Plug In Or Plug Out: Why Python & QGIS Can Lead the Way For Big Data

It's 2015 and we are well and truly walking head first into an information-driven world.  We capture data now at an alarming rate, increasing both the physical size and complexity of our datasets.  Our new hunger for data and data analysis has rendered many traditional data processing applications inadequate.  Whether you are a citizen or a data officer, you're likely to have had the same urges to analyse, capture, curate, search, and share information about the world around you...as a geo-visualisation of course.

In the world of modern mapping, this abundance of information has been very much welcomed.  More data equals more fuel to feed our cartographic minds.  'Big Data' and 'Linked Data' have become the talismans for how we try to make sense of this snow storm of information.  For us, this has been key to finding new patterns and unveiling new relationships as to how we as society interact with our environment.

It is with this open source community ‘pride’ that we are delighted to release our first plug-in for QGIS, with many more on the way in 2015 to help support our community mapping programme.
— Paul Georgie

Whilst both proprietary and open source mapping solutions have helped geographers and neo-cartographers make sense of quantitative fields in vector layers, there has been less tools available to allow us to make sense of qualitative information.

However, this is where the open source community takes the lead, and we are proud members of that community.  Whilst proprietary distributors strive to put the finishing touches on their new product, the face of open source is ever-changing.  The open source community works together to help make the most of the available tools, including how we can best utilise OpenStreetMap, QGIS and other open source platforms, to better our understanding of the world around us.  Python in particular has been a stand out language for enabling more effective (geo)processing of information, turning datasets into to more useable and (dare I say it) fun comprehensions of the digital world around us.

Our First Plug-In For QGIS Goes Live!

plugin-pic.jpg

By Steven Kay,

The aim of this plugin is to help visualise the contents of text fields in Vector layers.

I wrote this originally to help me visualise the wide, sparse text tables which are typical of OpenStreetMap data brought in via osm2pgsql. Such tables have dozens of fields, most of which are null for most records.

When planning mapping, it can be useful to know the discrete values for each field, and how often each value happens. This can also help with assessing the quality and consistency of the data.

When you choose from the list of Vector layers, the features in that layer and scanned and a list of text fields is shown below.

"Vector Text Field Statistics" is available now from the QGIS Plug In repository (still 'experimental' at point of publishing) and GitHub.