SGID Product Relaunch Update

In a June 2019 post “SGID Then and Now,” we touched on where we thought the SGID is going. Based on the survey responses from that article, we have changed direction in some areas and taken on new tasks in other directions. Now we’d like to share our vision, based on user input, for improving the SGID in general and relaunching the SGID product.

Redesigned SGID Open Data Website

Today we are releasing the redesign of our SGID Open Data, opens in a new tab website. We think the simple interface of the redesigned website and the new icon set will improve the user experience of searching, browsing, and exploring SGID data! We also updated the UGRC website icons to make the data navigation experience familiar between the two websites. Esri is working on enhancing the Open Data product, and we are following suit by putting a lot more effort into SGID Open Data into the future. The interface and icon set enhancements mark the beginning.

Open Data into the Future

Over the next few months, the download links on the UGRC website will be transitioning from static Google downloads to Open Data links. This will coincide with the authoritative data being shared through SGID Open Data. This adds new export formats (e.g., file geodatabase, shapefile, KML, and spreadsheet downloads), as well as feature service and GeoJSON APIs. The UGRC website will continue to remain the SGID data index where users can search and learn about SGID data, including data usage, descriptions, update cycles, and downloads for non Open Data content types.

Static SGID Data Layers

Another part of improving the SGID is assessing what information is still relevant to users and making sure that information is current. To surface the best data, we have trimmed down, opens in a new tab the number of tables that will reside in the SGID from over 500 to 320. The trimmed layers (i.e., data with historical relevance that are not the most current) are being placed in a special area, opens in a new tab in ArcGIS Online. These items can still be accessed using the shelved tag (e.g., tags:"shelved") but will not receive updates unless there is a specific business case. The convention is to store the most current data in the SGID with no date suffix and tag all prior vintages shelved with a year suffix (e.g., TemporalData would reside in the SGID while you would find TemporalData_1990 in ArcGIS Online). We think this organization will help separate the signal from the noise and provide users with the best search experience when using the SGID.

ArcGIS Online Cleanup

We have been using ArcGIS Online for organizations since its inception nearly a decade ago. A lot has changed and improved since then and our past efforts to make the most of this service have created some issues. To remedy those issues, we have correlated a list of all public layers owned by UtahAGRC with their usage stats to identify duplicate and ancillary data. We will be removing the lesser-used duplicates and nonauthoritative data. We hope to avoid disrupting web maps and items that use our data, but we need to tidy up to provide the best experience to our users.

New Data Editing Policy

UGRC continues to improve the way we serve our users. We are currently formalizing our policies on editing data in the SGID database in response to feedback from our users. All active editing will now be performed in an internal SGID database, and edits will be pushed to the public SGID database during off-hours. This will help reduce the risk of interrupting users while they are analyzing data within the time frame stated in our policy¹. These new data editing policies mark the maturation of the SGID as a database product and our commitment to treat the SGID as a production service.

Sweeper Project

Along with formalizing our data editing policy, UGRC has started the sweeper, opens in a new tab project to show our commitment to continually increase and stabilize the quality of SGID data. This open-source project gives us constant feedback about an increasing number of data quality metrics. All of the data being loaded into the internal SGID has passed every sweeper check, so all data entering the SGID will be issue free, and the checks are scheduled to run periodically to maintain this high quality of data.

SGID Data Life Cycle

At some point, datasets and applications reach the end of their usefulness. In an effort to stay focused on what is most productive, we occasionally deprecate products. We do our very best to notify our users via social media, opens in a new tab and our newsletter, opens in a new tab of these deprecations. The deprecations are tracked in an appropriately named repository, porter, opens in a new tab. In this repository, we track all of the dependencies and areas the product touches. We then triage these items to successfully deprecate the product. Please subscribe to updates for that repository, or at the very least subscribe to our newsletter to be informed about possible deprecations. Also, feel free to comment on deprecation issues if you will be affected by them.

Potential Google Cloud Managed PostGIS

UGRC is currently exploring the possibility of expanding the public database to users outside of the State Network with a Google Cloud managed PostGIS database. We are excited for this service since PostGIS can support the open-source GIS community and Esri clients². If this is something that is of interest to you, please send an email to Steve Gourley from UGRC at SGourley@utah.gov for a beta testing opportunity. These tests will help determine the feasibility of this solution.

Open Data Sharing

One of our main goals for the SGID moving forward is to continue to cultivate a sense of open data sharing in the Utah GIS community. The SGID Open Data website is a great example of the success of our data sharing workflow, as you can see from the authoritative data, opens in a new tab being shared from our agency partners (DNR Water Resources, DNR Water Rights, and Utah Division of State History). We invite our readers to join these partners and participate in this data-sharing initiative. Please get in touch with Zach Beck from UGRC at ZBeck@utah.gov | 801-538-3072, or Michael Foulger from UGRC at MichaelFoulger@utah.gov | 801-652-2258 if you have authoritative ArcGIS Online data to share to the SGID Open Data website.

We are excited for the future of the SGID, and we hope you will get involved to help us provide the highest quality GIS data to the State of Utah.