File-based data products in remain an important component of Utah’s State Geographic Information Database (SGID). In a change designed not to be noticed by users, UGRC recently moved the zipped shapefiles and file geodatabase SGID data from an FTP server to Google Drive, (or G Suite Drive if your prefer the latest rebranding from the folks in Mountain View).
- There were a few different reasons that drove the decision to move to Drive:
- Capabilities of Google Drive
- Diversifying the physical location or the data
- Simplifying upkeep
- Leveraging ‘unlimited storage’ of our Google Drive.
Data storage is never free and we want to make sure we are using it wisely. While the total compressed size of the SGID provided as static download files is not huge, the marginal cost of storage within UGRC’s existing Google Drive unlimited accounts is effectively zero. Moving to Drive will allow UGRC to eliminate some FTP storage costs while making the sharing of data easier and more organized.
Disaster recovery has always been a consideration in UGRC’s long-term planning and we feel it is important the data we store remain accessible in extreme circumstances. Our FTP servers have been reliable but, they have been based in the same two consolidated data centers that UGRC and other state agencies use for network drives and SDE database hosting. If a major earthquake or other disaster struck, GIS data for Utah is going to be a valuable resource in supporting the response and recovery period. And the first 72 hours are going to be the most critical. With SGID data resources discoverable and accessible from our github.io based web page (gis.utah.gov) and stored in Google’s geographically distributed data centers, GIS data users won’t be impacted by the health of the Utah data center and the immediate broadband network that services it.
FTP is old and sturdy with the original protocol written in 1971 and the current version dating to 1985. With that said, Google Drive (2012) has many useful capabilities. First and foremost Drive is compatible with the HTTPS specification which adds security and better integration with modern applications. Google also provides its own REST api, opens in a new tab for Drive that is great and free to use (within liberal quota limits). UGRC was able to use the Drive api client for Python, opens in a new tab to build an ‘update application’ that centralizes and automates our static zip file update process. The update application is in early development but the code and progress can be followed on the UGRC github organization, opens in a new tab.
If you regularly access data from gis.utah.gov you may have noticed that many of the links have already changed to Drive. In the near future, all of the file geodatabase and shapefile zip files will be linked to and stored on Drive. If you have any issues accessing the new links and storage please contact us. UGRC hopes that moving from FTP to Drive will continue to provide highly accessible GIS data resources, in a more resilient architecture, while making it easier for us to keep the latest data online.