SHIFTING YORKSHIRE’S ASSETS

30 August 2007, 8:49pm

Yorkshire Water predicted a 2 year programme would be needed to manually address their Positional Accuracy Improvement (PAI) issues. In the end the same results were delivered in just 13 days using automated tools and manual quality assurance. Yorkshire Water’s Mike Turner and Innogistic’s Kevin North explain how.

Positional Accuracy Improvement (PAI)

Positional Accuracy Improvement (PAI) refers to a programme of data quality enhancement undertaken by the Ordnance Survey (OS) between 2001 and 2006. It refers to the improvement of the position of features on OS rural mapping at scales of 1:2,500. Such features, whilst of high relative accuracy (i.e. in relation to one another), had poor absolute accuracy (i.e. position in the real world), due to the traditional surveying methods used at the time the data was captured. With the advent of Global Positioning Systems, absolute accuracy has become increasingly important, leading to OS re-issuing all affected maps over the five year period.

PAI at Yorkshire Water

With the position of the underlying map based shifting, PAI presented a challenge to Yorkshire Water, whose 60,000 kilometre network of clean and waste water assets have been accurately captured against OS base mapping in their corporate GIS. With a diverse geographic area, covering many rural areas, two thirds (12,000) of the 18,000 OS map tiles used by Yorkshire Water were affected by PAI. For an organisation with over 7 million assets recorded in its GIS, addressing PAI issues at Yorkshire Water potentially presented a significant challenge.

Researching the options

Yorkshire Water took the business decision to wait until embarking on their PAI programme until the Ordnance Survey had completed the improvement of all the tiles in its area. Yorkshire Water’s PAI Project team used this time to visit a number of other water companies to compare approaches.

As Mike Turner, Head of Asset Information at Yorkshire Water explains: “It looked like our options would be to either manually edit the data or to use an automated approach. The key lesson we learnt from elsewhere, was that existing automated techniques still needed extensive post process correction. At that stage, a manual correction programme looked like the only way we could deliver the quality needed”

Business plan & revised objectives

Having decided to go down the manual correction route, the PAI team put together a business plan to submit to Yorkshire Water’s board. The plan forecast a 2 year programme with a team of 7 people correcting the data. During this period, the bridge between transformed and untransformed data would need to be addressed by having dog-legs at the boundaries between the two.

Perhaps unsurprisingly the board rejected the business plan - a two year programme of works during which data quality would be affected (albeit temporarily) did not appeal. Not only that, but the board presented an ambitious set of objectives back to the team, namely:

1. To have completed their PAI programme within the next 9 months (including procurement of a solution)
2. To minimise disruption to GIS users and data maintainers
3. To have no deterioration (temporary or permanent) in data quality

The PAI team needed to re-visit the drawing board.

Piloting an Automated Approach

Yorkshire Water decided to take another look at options for an automated approach. They commissioned Innogistic Software to run a feasibility study on a pilot area late in 2006. The automated approach Innogistic adopted differed to other solutions available in the market. Instead of using rubber sheeting methods, its GeoFix tool made use of mathematical structures known as Voronoi diagrams, to calculate the transformation to be applied to the assets. Unlike other automated methods, the technique is not dependent on arbitrarily selected parameters and is a consistent data quality improvement technique which is particularly well suited to areas with irregularly distributed link points.

Dr Kieron Brown, Director of Production at Innogistic explains: “Academic research in Germany where they have had a similar PAI challenge has shown that algorithms based on the Voronoi technique deliver unambiguous results of a high quality. The feasibility study at Yorkshire confirmed this”.

Mike Turner confirms Brown’s comments: “Certainly on the basis of the feasibility study, we had a renewed faith in the potential for an automated method. A further attraction to the solution, was the technique could be used to process all of our data in one go, rather than in blocks or strips”.

Quality Assurance

Satisfied that an automated route could potentially deliver good quality results, Yorkshire Water nonetheless decided that manual quality assurance (QA) after the automated transformation would be key. They commissioned Innogistic to develop a powerful quality analysis and correction tool which used a rule base to highlight potential transformation errors to the editing team, for subsequent manual correction.

The test run

With the tools in place, a test run was conducted on all of the company’s data in April 2007. The test run verified that the approach was fully scaleable to all of Yorkshire Water’s assets, allowed valuable statistics and timings to be captured to plan the final operation and provided an opportunity to fine tune the quality assurance rules.

The final run

After the successful test run in April, Yorkshire Water and Innogistic planned the final operational run for June 2007. The test run had confirmed that the processing itself would take about 7 days on a cluster of 6 PC’s and a server, which if run over the weekend would only impact on five working days. On the 26th June, the operational data was checked out of Yorkshire Water’s live system and the GIS maintenance team sent on a five day training course so that no edits were made during the processing period. By the 4th July, the data had been returned to Yorkshire Water to start the quality assurance process.

Assessing the results

Of the 7 million assets in Yorkshire Water’s database, less than 1% (60,000) were highlighted as candidates for manual checking by the quality assurance tool. Candidates for checking were any features whose relationship to the underlying map features had changed during processing (e.g. features crossing where they had not previously). A GIS data maintenance team of 10 people set about the task of checking the results. It soon became clear that the majority of the candidates highlighted were not transformation errors – but had been flagged because the relationship to the underlying map was different on account of real world change (i.e. new features) on the base map itself. In fact of the 60,000 candidates, only 2400 – less than 0.05 % of Yorkshire Water’s entire asset base – were transformation errors which required manual correction. This high success rate meant that it only took the team just 8 working days to fully quality assure the data, rather than the 20 originally planned.

Conclusions

“The success of the PAI programme at Yorkshire Water has been unprecedented” comments Turner. “The fact that we only needed to manually intervene on a few thousand records of a total asset base of 7 million proves the success of the approach. To have processed and quality assured all our data in just 13 days, is an achievement which both Yorkshire Water and Innogistic Software should be rightly proud.”

Kevin North is Innogistic’s Business Development Manager for the Utilities sector
knorth@innogistic.co.uk

www.innogistic.co.uk

Kevin North