European Commission logo
INSPIRE Community Forum

It seems that there is only one fully compliant solution to serve multiple harmonised datasets trough WFS 2.0

We investigated several solutions to serve multiple harmonised datasets trough WFS 2.0 according to Technical Guidelines for Download Services. We are not discussing about providing access to only one dataset per server, but to provide access to more datasets.

 

Update 8 September 2016: The single compliant solution seems to be Snowflake GoPublisher WFS.

 

Update 9 May 2018: Because Stefania pointed in this tread that that Deegree is able to serve two different datasets for the same XML schema at different endpoints (asking for typeNames=ps:ProtectedSites in both cases) we started some tests.

So, it is still to see if Deegree is able to serve two different datasets based on multiple XML schemas at different endpoints in order to say that Deegree is compliant with Requirement 52, but is a big step forward in fullfiling the Requirement 52. For those data providers that need to serve feature types from one application schema at one endpoint Deegree is fulfiling the Req 52. We did not tested it yet to see if there are not other issues in order to confirm that it is able to serve multiple harmonised datasets based on multiple XML schemas trough WFS 2.0 according to Technical Guidelines for Download Services. Versions 3.4-RC (unstable) are passing the EPSG in URI format test, while 3.3 (stable) are not passing the test as EPSG is provided in URN format. As Deegree versions 3.4-RC are advertised as unstable and as there is no 3.4 stable version, it is not sure if Deegree is passing all the requirments of the Technical Guidelines for Download Services. However it should be noted that Deegree 3.4 version is OCG WFS 2.0 certified. If anyone had tested Deegree against each requirement from the TG, please share this information.

To conclude about Geoserver 2.13.0, for those data providers that need to serve feature types from one XML schema at one and only one endpoint, Geoserver 2.13.0 is fulfiling the Req 52. Geoserver seems not to be able yet to serve feature types from the same XML schema  at different endpoints, even if using the newly created isolated workspaces.

 

Till 8 September 2016 we thought that Go Publisher WFS (version 4.0.2) is not fullfiling the TG Requirement 46 ”Implementations shall conform to ISO 19142 Conformance Class Simple WFS”, more precise the WFS DescribeFeatureType request is not providing the expected result. Daniel Cocanu discovered that this is not related to GoPublisher WFS but it is related to XSD version 3.0 schemas.

 

The second most compliant solution seems to be Geoserver.  Unfortunately it does not fulfill at least one requirement, namely Requirement 52 ”A separate WFS endpoint shall be provided for each INSPIRE dataset thus providing one dataset per GetCapabilities response.” Tested on version 2.8.3. 

On third place seems to be Deegree (very good for testing purposes as it alows to serve datasets from the GML stored in memory) and on forth place ESRI ArcGIS for INSPIRE. We did not tested any other solution and we do not know any other solution (probably FME from SAFE can be a solution as well). 

Does anyone knows how to provide separate WFS endpoints for each INSPIRE dataset with Geoserver? 

Does anyone know other solutions to be tested (i.e.: FME from Safe) ?

Deoes anyone know other INSPIRE TG Requirements or Recommandations that cant be fullfiled by existing solutions ?

Isnt'it quite strange that Technical Guidelines are setting requirements that cant be fullfiled by any existing solution ?

Best regards,

Iurie Maxim

  • Iurie MAXIM

    Hello everybody,

     

    It seems that today the Geoserver bug that returned null values in complex features provided via WFS 2.0 was closed today in the 2.12-beta version, namely https://osgeo-org.atlassian.net/browse/GEOS-4773, as it was linked with another bug, namely https://osgeo-org.atlassian.net/browse/GEOS-8017

    Did not tested it yet, but if it works, than Geoserver will not return anymore ”null” instead of ”base” as it did till the fix:

    <gn:inspireId>

    <null:Identifier xmlns:null="http://inspire.ec.europa.eu/schemas/base/3.3">

    <null:localId>ROMAB0001</null:localId>

    <null:namespace>RO.ENV.GN.</null:namespace>

    </null:Identifier>

    </gn:inspireId>

     

    These are good news, but still there are important issues to be solved in Geoserver in order to allow its use for implementing INSPIRE.

     

    Another important bug that need to be solved is this one: https://osgeo-org.atlassian.net/browse/GEOS-5512

     

    Best regards,

    Iurie Maxim

     

  • Iurie MAXIM

    Hello everybody,

    We found that even if the issue https://osgeo-org.atlassian.net/browse/GEOS-4773 appear to be solved in 2.12-beta version, it was solved only for simple features and not for complex features.

    Therefore null namespace prefixes are still present in virtual services for complex features. As all features in INSPIRE are complex features it means that unfortunately there are no good news yet for data providers counting on Geoserver.

    Making more investigations we found that null namespace prefixes are provided while making some other WFS requests as well.

    To be noted that complex features are handled in Geoserver trough app-schema, therefore the issues can be related to Geoserver, or to app-schema, or to both.

    Probably all or some of the issues described below are related to the fact that Geoserver creates so called “workspaces” that are in a one-to-one relationship with a XSD schema and therefore nor a complex features and nor a dataset composed of more complex features are not managed in a single workspace.
    I found a discussion related to this topic here: http://osgeo-org.1560.x6.nabble.com/Allowing-multiple-workspaces-to-use-the-same-name-space-URI-td5307302.html

    The following WFS requests are returning null namespace prefixes:

    1. A GetPropertyValue request to retrieve from a complex feature type (based on multiple XSD files and therefore stored in multiple workspaces) the value of a property that is found in another workspace than the root of the complex feature type is returning null namespace prefixes.

    In order to implement INSPIRE, a protected site complex feature type is based on three XSD schemas, namely ps.XSD (protected site), base.XSD (it includes the inspireId) and gn.XSD (geographical name), therefore three workspaces are created from one complex feature type: ps, base and gn.

    If a user wants to retrieve the inspireIds or the geographical names of all or some protected sites, unfortunately the GetPropertyValue request returns null namespace prefixes.

    Links to test nulls:

    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=GetPropertyValue&typeName=ps:ProtectedSite&valueReference=ps:inspireID/base:Identifier&count=2

    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=GetPropertyValue&typeName=ps:ProtectedSite&valueReference=ps:siteName/gn:GeographicalName/gn:spelling/gn:SpellingOfName&count=2

    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=GetPropertyValue&typeName=ps:ProtectedSite&valueReference=ps:legalFoundationDocument/gmd:CI_Citation&count=1

    To be noted that a GetPropertyValue request does not return null namespace prefixes for the first nested element:

    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=GetPropertyValue&typeName=ps:ProtectedSite&valueReference=gml:identifier&count=2

    but it returns null namespace prefixes for the second, third, .. nested element:

    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=GetPropertyValue&typeName=ps:ProtectedSite&valueReference=ps:siteName&count=1

    However, a GetFeature request is not providing null namespace prefixes for that complex feature

    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=GetFeature&typeName=ps:ProtectedSite&count=1

    Other software is not returning null namespace prefixes in GetPropertyValue requests, therefore there is not a WFS request issue.

    2. A GetFeature request in order to retrieve a dataset trough a Stored Querry is providing null namespace prefixes while retrieving a dataset with multiple featureTypes.

    In practice a dataset contains more than one featureType from multiple INSPIRE data themes, all having the same metadata and having the geometries in topology, being produced at the same precision and having the same lineage.

    In this example the dataset contains protected sites (ps), administrative units (au), bio-geographical regions (br) and their corresponding geographical names (gn), for all these being created five so called workspaces: ps, au, br, gn and base.

    A GetFeature request in order to retrieve this dataset trough a Stored Querry is returning null namespaces.

    Link to test nulls:

    http://inspire.teamnet.ro/geoserver/wfs?request=GetFeature&StoredQueryID=http://inspire.ec.europa.eu/operation/download/GetSpatialDataSet&version=2.0.0&crs=epsg:3035&count=1

    Other software is not returning null namespace prefixes, therefore there is not an WFS request issue.

    3. A GetFeature request to a virtual service (in this case "ps") are returning null namespace prefixes.

    Link to test nulls:

    http://inspire.teamnet.ro/geoserver/ps/wfs?request=GetFeature&version=2.0.0&typeNames=ps:ProtectedSite&count=2

    Other software is able to create separate endpoints per each dataset even if the dataset has more than one complex featureType.

    General considerations:

    INSPIRE requires data providers to have unique endpoints for each dataset. Therefore in INSPIRE one datset = one endpoint.

    Currently the unique solution in Geoserver to provide multiple endpoints is the use of virtual services. In Geoserver virtual services endpoint names are inheriting the names of the workspaces which unfortunately are inheriting the name of the XSD schema.

    Because a complex feature is based on multiple XSD schemas (all INSPIRE complex features are based on base.XSD schema as well) in order to serve an INSPIRE complex feature in Geoserver there will be at least two workspaces, one named base and another named similar to the data theme (ex: ps for protected sites) Most complex features have geographical names, and gn.XSD is used in those complex feature types. In this case a data provider can, and therefore he should provide access at two complex features, one beeing gn:GeographicalName and the other of the data theme (i.e.: ps:ProtectedSite). In this case already the dataset is composed of two feature types (i.e: ps and gn) and it is based on three XSD files (including base).

    Therefore, for complex features, the so called workspaces in Geoserver are not workspaces as those entities can be considered only as XSD holders.

    For complex features a workspace should be based on multiple XSD schemas, therefore a one-to-many relationship should exist between a workspace and the XSDs and not a one-to-one relationship as it is now.

    As INSPIRE requires a unique endpoint per dataset, so, from an INSPIRE data provider point of view the workspace that is used for creating virtual services should store the dataset. Each such new workspace should have a one-to-many relationship with al XSD schemas used by those featureTypes that are part of the dataset. The endpoint used for the virtual service should provide access to one, many and altogether featureTypes that are part of the dataset.

    From an INSPIRE data provider perspective a workspace is tight to a dataset. A dataset can contain more than one complex feature type, therefore a dataset is based on more than one XSD schema and more than one complex featureType. A dataset is similar to a "feature dataset" in ArcGIS that contains "feature classes" that are similar to featureTypes (each featureType can be represented in different styles based on a SLD, for each style being created a WMS “layer”).

    If a single instance of Geoserver would be able to provide access to a single dataset, than workspaces do not make sense from INSPIRE perspective as they are not actually workspaces for complex features.

    If a single instance of Geoserver would be able to provide access to more datasets, then one workspace may be linked to the one dataset based on multiple featureTypes and therefore on multiple XSD schemas. As INSPIRE requires datasets to be accessible at a unique endpoint, virtual services for such workspaces make sense.

    We came as well to the conclusion that with the current Geoserver configuration WFS and WMS should be accessible from different endpoints (probably even from different Geoserver instances), but this is another subject, maybe to be further discussed and it is linked to poor performance of rendering images based on complex features, so flat simple features are more suitable for WMS. The link between WMS and WFS features is ensured trough short URL, such as for example http://inspire.teamnet.ro/ENV/PADS/psPS/RONPA0022

    In this context a new Geoserver bug was opened here https://osgeo-org.atlassian.net/browse/GEOS-8108

    Iurie Maxim
    Teamnet
    Romania

     

  • Iurie MAXIM

    On the other hand, there are good news, as the bug https://osgeo-org.atlassian.net/browse/GEOS-5512 can be closed.

    There is a way of querring multiple feature types trough one stored querry discovered by my colleague Sorin Rusu.

    See the example of the stored querry with the Id "http://inspire.ec.europa.eu/operation/download/GetSpatialDataSet"&nbsp;

    while making the DescribeStoredQuerries request on our server:
    http://inspire.teamnet.ro/geoserver/wfs?version=2.0.0&request=DescribeStoredQueries

    Iurie

  • Stefania MORRONE

    By Stefania MORRONE

    Hi all,

    find more discussion on Req. 52 (providing separate wfs endpoint for each dataset) in this TC discussion

    Best,

    Stefania

  • Iurie MAXIM

    Hi,

     

    As we tested quite a lot the Geoserver 2.13.0 to understand how the new isolated workspaces can be used for implementing INSPIRE and we came to certain conclusions that we would like to share.

    I. A GetFeature Request assumes that both the namespace ,actually the XML namespace = prefix and the featureType is provided: “request=GetFeature&typeNames=namespace:featureType”
    The namespace used in the GetFeatureRequest is the XML namespace
    For example:
    If namespace URI= “target schema”= https://inspire.ec.europa.eu/schemas/ps/4.0/ProtectedSites.xsd, for the ”ProtectedSite” featureType the XML namespace (prefix) is ”ps” as declared in the schema xmlns:ps="http://inspire.ec.europa.eu/schemas/ps/4.0&quot;.
    So there is a namespace URI and an XML namespace.
    Currently Geoserver is using the name of the workspace *as *prefix (XML namespace), so the GetFeature request works only if the typeNames=workspace:featureType and this is misleading and also not correct according to INSPIRE
    As we created a workspace named “CDDA” and in the “namespace URI” we provided the “URI of the target schema” = https://inspire.ec.europa.eu/schemas/ps/4.0/ProtectedSites.xsd. In order to retrieve the features, the GetFeature request will contain “typeNames=CDDA:ProtectedSite”.

    However the request should contain “typeNames=ps:ProtectedSite” as the ”XML namespace” = ”prefix of the featureType” should be ”ps” according to INSPIRE.

    Our understanding was that the isolated workspaces were created in order “to be able to publish the same feature type (name space + name) multiple times”. However, namespace should refer to both namespace URI and XML namespace (prefix), while currently it refers only to namespace URI.

    In the app-schema there are both the prefix (XML namespace) and the uri (namespace URI) used for mapping as can be seen bellow.

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <ns3:AppSchemaDataAccess xmlns:ns2="http://www.opengis.net/ogc&quot; xmlns:ns3="http://www.geotools.org/app-schema"&gt;
    <namespaces>
    <Namespace>
    <prefix>ps</prefix>
    <uri>http://inspire.ec.europa.eu/schemas/ps/4.0</uri>
    </Namespace>

    Therefore, as currently implemented, isolated workspaces should be used in conjunction with virtual services in order to be able to publish the same featureType ”ps:ProtectedSite” multiple times. With current implementation it seems that we can publish the the ProtectedSite featureType but with different *prefixes *that are now inherited from the “workspace name”. So we can publish at the “ps” virtual service endpoint the “ps.ProtectedSite” featureType and at the ”CDDA” virtual service endpoint we can publish ”CDDA:ProtectedSite” even if we need to serve ”ps:ProtectedSite” trough both virtual service endpoints.
    We see that there is a problem that the workspace name that is used for virtual services is inherited as well by the namespace of the featureType (prefix (XML namespace)) and this is not usable for INSPIRE.

    Therefore, we are proposing to change the workspace UI and the functionalities behind.

    Now the workspace UI has two elements:
    name” that is storing the “name of the workspace”
    namespace URI” that is storing the “URI of the target schema”

    In the UI should be added as well the
    XML namespace (prefix of the featureType)” in order to store the namespace of the featureType that is used in the GetFeature request, namely the prefix of the featureType.

    As an example:
    workspace name” = ”CDDA”
    namespace URI” = https://inspire.ec.europa.eu/schemas/ps/4.0/ProtectedSites.xsd
    XML namespace” = ”ps” (see https://inspire.ec.europa.eu/schemas/ps/4.0/ProtectedSites.xsd where xmlns:ps="http://inspire.ec.europa.eu/schemas/ps/4.0&quot;)

    The ”XML namespace” should not be taken anymore from ”workspace name” as currently implemented and In this case we would be able to request the ”ps:ProtectedSite” at the ”CDDA” virtual service endpoint.
    With this approach, a data provider will be able to serve different ”ps:ProtectedSite” at different endpoints. Each endpoint will be linked to a dataset containing feature types that are based on a single XSD schema.

    II. To complicate the things a little bit, if we would like to take into consideration that a dataset can be formed by multiple featureTypes from different XSD schemas, than each endpoint should be linked to a dataset containing feature types that are coming from more than one XSD schema.
    In a such example in a
    ”workspace name” = ”CDDA”
    ”namespace URI(s)” = https://inspire.ec.europa.eu/schemas/ps/4.0/ProtectedSites.xsdhttps://inspire.ec.europa.eu/schemas/gn/4.0/GeographicalNames.xsd
    ”XML namespace(s)” = ”ps”, ”gn”
    workspace CDDA endpoint should allow direct access at both ps:ProtectedSites and gn:GeographicalNames featureTypes.

    III. Not sure if Geoserver was built in such way to allow INSPIRE implementation, but currently it seems that Geoserver has the following limitations in relation with INSPIRE:
    1. In a Geoserver instance it is not possible to serve more than once the same featureType, including its prefix (XML namespace), even if using isolated workspaces and virtual services endpoints. In other words it is possible to serve a feature type only by using different prefix (i.e.: ps.ProtectedSite and ps1.ProtectedSite). The prefix is inherited from the name of the workspace. This breaks several INSPIRE requirements and the only solution is to install separate instance of Geoserver for each feature type that need to be published several times.
    2. In a Geoserver instance is not possible to serve more than one dataset if that dataset is based on more than one XSD schema and more featureTypes need to be served at a certain endpoint. In other words it is possible to serve a dataset composed by ps:ProtectedSites features and their corresponding gn:GeosgraphicalNames features at a certain endpoint, but it is not possible in the same instance to serve as well at another endpoint a dataset composed of au:AdministrativeUnits and their corresponding gn:GeograpficalNames features.
    3. It is not yet WFS 2.0.0 nor WMS 1.3 compliant and is not passing the CITE OGC Validations (strangely it fails even the GetFeatureById WFS request with app-schema image GEOS-6233 OPEN ).

    I added a bug on Geoserver with the same content https://osgeo-org.atlassian.net/browse/GEOS-8734

    Hope it helps,
    Iurie Maxim
    http://essensys.ro

  • Stefania MORRONE

    By Stefania MORRONE

    Regarding the fulfilment of requirement 52 - having one dataset per GetCapabilities response - I would point out that this issue is not present in deegree: if I want to serve e.g. Natura 2000 SPA and Natura2000 SCI datasets with same deegree instance, I am able to just configure two different endpoints (using the same instance and workspace) and typeNames=ps:ProtectedSite in both cases.

    Please follow these links (be aware that this is a test server):

    http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/ps_Natura2000SPA?request=GetFeature&Language=eng&service=WFS&version=2.0.0&count=10&typeNames=ps:ProtectedSite

    http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/ps_Natura2000SCI?request=GetFeature&Language=eng&service=WFS&version=2.0.0&count=10&typeNames=ps:ProtectedSite

    Stefania

  • Iurie MAXIM

    Hi Stefania,

    Any chance to present more than one feature type at one endpoint ?

    So can you confirm if it is possible to access two featureTypes at the ps_Natura2000SPA endoint, for example ps:ProtedtedSite and the corresponding gn:GeograpficalName ?

    We can consider that a Natura 2000 SPA Dataset is formed by these two featureTypes. At least for WMS is important to see the labels of the sites and those labels are shown trough gn:GeographicalName portrayal, so at least these two featureTypes should be part of the same dataset.

    It seems that is possible such a one to many relationship with Degree as it was possible with Snowflake Go Publisher WFS as well few years ago. With Geoserver clearly there are no chances now to do this in the same instance.

    Iurie

  • Iurie MAXIM

    I updated the document with the following

    Update 9 May 2018: Because Stefania pointed in this tread that that Deegree is able to serve two different datasets for the same XML schema at different endpoints (asking for typeNames=ps:ProtectedSites in both cases) we started some tests.

    So, it is still to see if Deegree is able to serve two different datasets based on multiple XML schemas at different endpoints in order to say that Deegree is compliant with Requirement 52, but is a big step forward in fullfiling the Requirement 52. For those data providers that need to serve feature types from one XML schema at one endpoint Deegree is fulfiling the Req 52. ‚ÄčWe did not tested it yet to see if there are not other issues in order to confirm that it is able to serve multiple harmonised datasets based on multiple application schemas trough WFS 2.0 according to Technical Guidelines for Download Services. Versions 3.4-RC (unstable) are passing the EPSG in URI format test, while 3.3 (stable) are not passing the test as EPSG is provided in URN format. As Deegree versions 3.4-RC are advertised as unstable and as there is no 3.4 stable version, it is not sure if Deegree is passing all the requirments of the Technical Guidelines for Download Services. However it should be noted that Deegree 3.4 version is OCG WFS 2.0 certified. If anyone had tested Deegree against each requirement from the TG, please share this information.

    Iurie

  • Iurie MAXIM

    I noted that right from the beginning of the thread we pointed an issue with Deegree that is still present at least in the 3.3.18 version provided for test by Stefania.

    So, it is this a correct behavior of Deegree for DescribeFeatureType KVP requests ?

    http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/ps_Natura2000SCI?service=wfs&version=2.0.0&request=DescribeFeatureType&typename=ps:ProtectedSite

    When making a DescribeFeatureType request in the browser (KVP request) Deegree is providing a file for download with no filename associated to it. All other software are returning the result in the browser. There is a MIME type issue or what ?
     
    Iurie

     

     

  • Iurie MAXIM

    It seems that is a correct behavior:

    For DescribeFeatureType, GetPropertyValue, GetFeature, GetFeatureWithLock the default OUTPUTFORMAT value is "application/gml+xml; version=3.2" indicating that resources in the response document shall be encoded using GML (see ISO 19136:2007).

    So not only GetFeature response should be encoded as GML, but DescribeFeatureType and GetPropertyValue as well.

    See http://www.opengeospatial.org/standards/wfs

    However the Deegree testing instance of Stefania is not configured to provide a GetFeature response as MIME Type application/gml+xml, while it is configured to provide a DescribeFeatureType response as MIME Type application/gml+xml, creating confusion.

    Discovered this thread as well in regards with Deegree 3.3 versions and this subject.

    Iurie

     

This discussion is closed.

This discussion is closed and is not accepting new comments.

Biodiversity & Area Management

Biodiversity & Area Management

If themes like Protected Sites, Area Management/Restriction/Regulation Zones and Reporting Units, Habitats and Biotopes, Species Distribution, Bio-geographical Regions matters to you, join these groups!