October / November ’15 Report from the Data Infrastructure Team

Hi Folks,

Welcome to our new regular report which will capture a broader work program including: data.gov.au; the NationalMap; the work we are doing on the Open Data Network; data skills and capability development across the APS; and any other news from the rest of the Public Data Branch.

Our team has a new name (Data Infrastructure and Government Engagement) within the Public Data Branch at PM&C. We also have several new members and some new responsibilities (like the NationalMap). You’ll be seeing more about the NationalMap in the future as we integrate it into our regular reporting.  We’ve already enabled it as the primary visualisation tool for spatial data on data.gov.au. Below you’ll find an example of the Native Title Determinations Outcomes dataset.

an image demonstrating the native title determinations outcomes dataset visualized on the nationalmap

 

The data publishers amongst our readers will be keen to read up on csv-geo-au. A specification for displaying Australian spatial data captured in tabular datasets.

Speaking of previews, we’ve just introduced a new plugin for data.gov.au that lets you peek at the content of uploaded zip files. This includes those not physically hosted on data.gov.au. We think you’ll find this a helpful way to quickly identify interesting and useful data within zip files.

a preview of how the zip viewer functionality works

The view will be automatically added to any new zip file uploaded to data.gov.au. The data custodian can add the view to existing resources by:

  1. Browsing to the zipped resource
  2. Clicking the Manage button
  3. Clicking on the Views tab link
  4. Clicking on the New view button and select ZIP viewer
  5. Entering an appropriate Title (we suggest Zip view)
  6. Clicking the Add button

We are also developing a plugin for data.gov.au that automatically extracts and hosts machine readable data from zip files making them API enabled.

Unpublished Datasets

You may have noticed unpublished datasets being added to data.gov.au in the last few weeks.  These records are pointers to Australian Government data holdings that may have not been released yet or have additional usage caveats. The listing of this data on data.gov.au makes it easier to discover this data and improves user’s knowledge about data holdings. Like, for example, did you know that the Attorney General’s Department has a register of marriage celebrants?  To access these datasets users should get in touch directly with the data custodian in the first instance. The contact point for each dataset is listed as part of its metadata (located under the Additional Info heading).

You may have noticed a drop in the dataset count in the last few weeks. This is down to a number of datasets being duplicated because of a harvesting issue. We’re exploring ways of mitigating this duplication going forward.

Skills Calendar and Data Roles

We are also pulling together a list of data related courses (e.g. data analysis, statistics, R, python) and putting them into a training calendar for 2016. Send us an email if you have or know of any short courses that you think the wider APS should know about. 

We are also working to define some agreed data roles and competencies. What we have found is that a good data analyst is someone with a composite of many skill sets. Working closely with a number of agencies we are hoping to provide clarity around the data roles required to support the functions of government.

NationalMap Forum

We held a NationalMap Forum on 24 November to facilitate an understanding of the NationalMap as a tool to visualise and embed geospatial data and for users to exchange information and experiences. Presentations covered the current version of the NationalMap, upcoming modifications, data related matters and other maps built on the NationalMap software. The forum also provided an opportunity for attendees to voice their current imperatives and future needs.

We will be holding a similar workshop for the data.gov.au platform on Wednesday, 16 December. There will be two sessions, both intended to set an environment for informal dialogue.

Data.gov.au Forum

Session 1 – Presentations: 9.00am – 11.00am

  • Presentations will be from the Department of the Prime Minister and Cabinet (Public Data Branch) and guest agencies will cover:
    • The Public Data Agenda and strategic approach to data
    • Introduction to and the future of data.gov.au
    • Using Data to Inform, Analyse and Advise
    • Publishing for Efficiency
    • Automating Data
    • Value of Data
  • Opportunity for those involved in policy / project management roles to provide feedback and discuss future needs.

Session 2 – Workshop: 11.00am – 1.30pm

  • Interactive small group technical discussions with the data.gov.au team and Data61 to provide information / assistance on formatting and infrastructure issues.
  • Divided into small group workshops:
    • Using data.gov.au and publishing data for beginners
    • Advanced technical questions on data.gov.au including automating data publishing
    • Feature requests and building a roadmap for data.gov.au
  • There will be a concurrent session being run in a different room with members of the general public that use government data to identify data user needs.
  • At 12:30 the workshops will wrap up and there will be an informal networking lunch where government and non-government participants can meet and greet to discuss data. If you do not wish to attend the lunch session, all formal workshop activities will be finalised by 12:30.

If you’d like to come along please send us an email to data [at] pmc.gov.au.

Cheers,

Allan
Data Infrastructure and Government Engagement team, Public Data Branch

  October November
Total Webpage Visits 33,447 33,915
Total Pageviews 136,963 141,936
Total Discoverable Datasets 7,163 7,209
Total Organisations 169 170
Total Data Resources / Files 34,525 34,744
Total Machine Readable / Data API Resources 3,777 3,793

 

October
Datasets
October
Resources
October
datastore_sql_search
Energy Rating Data for household appliances - Labelled Products Air Conditioners 1,795
ASIC - Company Register Company Register - Current 963
Energy Rating Data for household appliances - Labelled Products Fridges and Freezers 608
Energy Rating Data for household appliances - Labelled Products Televisions 558
Energy Rating Data for household appliances - Labelled Products Clothes Dryers 454
Energy Rating Data for household appliances - Labelled Products Dishwashers 402
Energy Rating Data for household appliances - Labelled Products Computer Monitors 363
Australian Public Holidays Dates Machine Readable Dataset Most Recent Update 85
Historical Australian Government Contract Data 2014-2015 Commonwealth Government Contract Data 53
Commercial Building Disclosure CBD Downloadable Data Set 51
November
Datasets
November
Resources
November
datastore_sql_search
Energy Rating Data for household appliances - Labelled Products Air Conditioners 990
Energy Rating Data for household appliances - Labelled Products Fridges and Freezers 506
Energy Rating Data for household appliances - Labelled Products Televisions 485
Energy Rating Data for household appliances - Labelled Products Dishwashers 389
Energy Rating Data for household appliances - Labelled Products Clothes Dryers 351
Energy Rating Data for household appliances - Labelled Products Computer Monitors 350
Energy Rating Data for household appliances - Labelled Products Company Register - Current 265
ASIC - Company Register Company Register - Current 151
School Locations (Victoria) all schools list.csv 32
Historical Australian Government Contract Data 2014-2015 Commonwealth Government Contract Data 27

 

5 Most Active Organisations

Top 5 Organisations by Total Datasets

 5 Most Recent Published Datasets

5 Most Recently Updated Datasets

 

Most Highly Requested Data Sets Votes Status
Free the G-NAF Address Dataset 156 In Review
Free Postcode and Postal Address Data 101 In Review
NBN Datasets 53 In Review
DCDB Cadastre 45 In Review
Australians Schools 43 In Review

Leave a Comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.