European Commission logo
INSPIRE Community Forum

Download Service WFS: StoredQuery and ResponsePaging for large datasets?

By Thijs BRENTJENS Replies (2)

Hello all,

In the Netherlands we have several large datasets served by WFS to the INSPIRE network. One of the TG requirements for WFS is to provide a StoredQuery to download the entire dataset. For large datasets this is a practical challenge: users could (accidentally or intentionally) download an entire dataset, maybe GBs, with a single request to a WFS, which could freeze applications. For the service provider this may cause problems too on their infrastructure.

For datasets with a high update frequency it is not an option to provide pre-processed / cached datasets and/or use ATOM feeds as Download service. So serving them with WFS directly is sometimes the only (good) option. One approach is to add a (low) serverside limit to the maximum number of features to return. E.g. 1000 features at once. A user then could use WFS Response Paging to fetch the other features, if needed. This works on the INSPIRE StoredQuery too, we've tested this, also with WFSes offering more than 1 featuretype.

However Response Paging is not mentioned in the Technical Guidance for Download Services. Clients have more and more support for Response Paging.

What do you think of this approach? Is this approach in line with Download Services?

And is this an issue other organizations have too? If so, what solution(s) are you using?

  • Jari REINI

    Hi Thijs,

    I agree, response paging is currently the only good option. We are also using paging and my interpretation is that it is ok.

    Nvertheless, we must bring up this question in the next MIG-T meeting.



    By Thijs BRENTJENS

    Hi Jari,

    Thanks, good to know you are doing this too and indeed, it should be discussed in the next MIG-T. I'll ask if a colleague will attend and bring this up. Or will you attend?