Importing Data Just Got a Whole Lot Faster...for Open Performance Too!
We recently made some important import performance improvements to the Socrata Platform, and while we initially published the update in an Open Data release note, the benefits are also for Open Performance sites!
Despite the rapid increase we’ve seen in automating data updates, many datasets are still imported by users logging into their Socrata platform and using the import/update wizard. During import, processing the data file could take a while, especially for very large datasets.
To reduce the amount of time spent waiting patiently for data to load, and get data publishers quickly moving on to more high-value tasks like data visualization and analysis, we are excited to announce performance improvements to the Socrata import wizard. Importing data through the import wizard is now up to 9 times faster than previous import times!
To provide some examples, we profiled the performance across 10 different files and the improvement is shown below. Keep in mind that import speeds can vary across datasets and is still dependent on a variety of factors besides the shape of the data.
By moving data through the import process quickly, data publishers will be whisked from step to step, leaving little time for coffee breaks or jumping to other tasks. So fuel up before importing data, and happy publishing!
Small Updates & Bug Fixes
- Updates and fixes for Goal Inventory
- Open Performance APIs now support HTTP Basic authentication.
- Added validation to the quick edit modal to ensure the start date for a goal is chronologically before the end date.
- Built a loading spinner into the goal page; it is displayed while the list of goals is being populated.
- Clicking Feedback and choosing to include a screenshot no longer results in an error.
- Fixed text wrapping in the dashboard column to behave like the rest of the columns.
- Fixed a styling issue for the Microsoft Edge browser.
- Fixes to goal pages and dashboards
- Fixed an issue where some dashboards were disregarding the user-defined sort order of the categories and rendering the default sort order.
- Fixed an issue in the goal page narrative where publishers on some domains couldn’t choose a new dataset to replace an existing visualization on the page.