In a June 2019 post “SGID Then and Now,” we touched on where we thought the SGID is going. Based on the survey responses from that article, we have changed direction in some areas and taken on new tasks in other directions. Now we’d like to share our vision, based on user input, for improving the SGID in general and relaunching the SGID product.
Redesigned SGID Open Data Website
Today we are releasing the redesign of our SGID Open Data website. We think the simple interface of the redesigned website and the new icon set will improve the user experience of searching, browsing, and exploring SGID data! We also updated the AGRC website icons to make the data navigation experience familiar between the two websites. Esri is working on enhancing the Open Data product, and we are following suit by putting a lot more effort into SGID Open Data into the future. The interface and icon set enhancements mark the beginning.
Open Data into the Future
Over the next few months, the download links on the AGRC website will be transitioning from static Google downloads to Open Data links. This will coincide with the authoritative data being shared through SGID Open Data. This adds new export formats (e.g., file geodatabase, shapefile, KML, and spreadsheet downloads), as well as feature service and GeoJSON APIs. The AGRC website will continue to remain the SGID data index where users can search and learn about SGID data, including data usage, descriptions, update cycles, and downloads for non Open Data content types.
Static SGID Data Layers
Another part of improving the SGID is assessing what information is still relevant to users and making sure that information is current. To surface the best data, we have trimmed down the number of tables that will reside in the SGID from over 500 to 320. The trimmed layers (i.e., data with historical relevance that are not the most current) are being placed in a special area in ArcGIS Online. These items can still be accessed using the
shelved tag (e.g.,
tags:"shelved") but will not receive updates unless there is a specific business case. The convention is to store the most current data in the SGID with no date suffix and tag all prior vintages
shelved with a year suffix (e.g., TemporalData would reside in the SGID while you would find TemporalData_1990 in ArcGIS Online). We think this organization will help separate the signal from the noise and provide users with the best search experience when using the SGID.
ArcGIS Online Cleanup
We have been using ArcGIS Online for organizations since its inception nearly a decade ago. A lot has changed and improved since then and our past efforts to make the most of this service have created some issues. To remedy those issues, we have correlated a list of all public layers owned by UtahAGRC with their usage stats to identify duplicate and ancillary data. We will be removing the lesser-used duplicates and nonauthoritative data. We hope to avoid disrupting web maps and items that use our data, but we need to tidy up to provide the best experience to our users.
New Data Editing Policy
AGRC continues to improve the way we serve our users. We are currently formalizing our policies on editing data in the SGID database in response to feedback from our users. All active editing will now be performed in an internal SGID database, and edits will be pushed to the public SGID database during off-hours. This will help reduce the risk of interrupting users while they are analyzing data within the time frame stated in our policy1. These new data editing policies mark the maturation of the SGID as a database product and our commitment to treat the SGID as a production service.
Along with formalizing our data editing policy, AGRC has started the sweeper project to show our commitment to continually increase and stabilize the quality of SGID data. This open-source project gives us constant feedback about an increasing number of data quality metrics. All of the data being loaded into the internal SGID has passed every sweeper check, so all data entering the SGID will be issue free, and the checks are scheduled to run periodically to maintain this high quality of data.
SGID Data Life Cycle
At some point, datasets and applications reach the end of their usefulness. In an effort to stay focused on what is most productive, we occasionally deprecate products. We do our very best to notify our users via social media and our newsletter of these deprecations. The deprecations are tracked in an appropriately named repository, cemetery. In this repository, we track all of the dependencies and areas the product touches. We then triage these items to successfully deprecate the product. Please subscribe to updates for that repository, or at the very least subscribe to our newsletter to be informed about possible deprecations. Also, feel free to comment on deprecation issues if you will be affected by them.
Potential Google Cloud Managed PostGIS
AGRC is currently exploring the possibility of expanding the public database to users outside of the State Network with a Google Cloud managed PostGIS database. We are excited for this service since PostGIS can support the open-source GIS community and Esri clients2. If this is something that is of interest to you, please send an email to Steve Gourley from AGRC at SGourley@utah.gov for a beta testing opportunity. These tests will help determine the feasibility of this solution.
Open Data Sharing
One of our main goals for the SGID moving forward is to continue to cultivate a sense of open data sharing in the Utah GIS community. The SGID Open Data website is a great example of the success of our data sharing workflow, as you can see from the authoritative data being shared from our agency partners (DNR Water Resources, DNR Water Rights, and Utah Division of State History). We invite our readers to join these partners and participate in this data-sharing initiative. Please get in touch with Zach Beck from AGRC at ZBeck@utah.gov or 801-538-3072, or Michael Foulger from AGRC at MichaelFoulger@utah.gov or 801-652-2258 if you have authoritative ArcGIS Online data to share to the SGID Open Data website.
We are excited for the future of the SGID, and we hope you will get involved to help us provide the highest quality GIS data to the State of Utah.