The Geodata subsystem

Configuration of the geodata subsystem is in the table "manage_geodata" in the file /cgi-bin/program_data.sqlite

This table is structured as follows:

field	a name for the field - should suit the data - for example "postal_code" or "osgrid"
flags_JSON	an extensible set of values that control processing of data - see below

flags_JSON

calculate	boolean	whether the value can be calculated now or should be fetched in the background
extend_cache_by	integer	how many days to keep the data before refetching it
depends_on	fieldname	this is fetched at the same time as fieldname

There is a table in the master data – data/canal.sqlite – called "geodata_for_place" as follows:

place_id	the place this data belongs to
fields_JSON	the value for every field from the manage_geodata table
cache_JSON	for each field, when it was last fetched
recache_on	the earliest of all these values - for quick and easy checking

Whenever a page is displayed in the place gazetteer, the page cgi-bin/includes/geodata.can is included and the function "Get_Place_Geodata" is called. This works through each record in the manage_geodata table and checks that it has a value (new fields could have been added) and that it should not be refreshed based on the cache information.

Geodata is actually generated by files in cgi-bin/includes/geodata_plugins/ For all rows in manage_geodata that don't have the depends_on flag set, there should be a file named calculate_ or fetch_ (depending on whether the calculate flag is set) followed by the field name (for example calculate_osgrid.can or fetch_address.can).

This takes the placeinfo as a lookup parameter. Things being fetched also take the extra_JSON field as a second parameter and should use this to check that the fieldname_override flag isn't set before returning a new value.

To do

Batch move these columns from place_extra and place_geodata

• postcode
• roadname
• roadnumber
• address

Code things to do

• There's an existing geodata in includes/cached_gaz_data.can
• There's the opencage_geodata.can file in includes as well
• Integrate streetmap postcode (it's better than opencage for remote places)
• NB: this only works for grid refs, so use open cage for outside the UK
• Remove current postcode fetching from do_queued_action and what calls it now it's integrated
• What happens for OS grid when out of range?
• Need change trail from revision table
• remove columns from place_extra
• remove place_geodata table
• there is a geodata search in placefinder that still seems to use Google
• there is postcode matching in placefinder
• need a generic routine to get the best postcode (using ukpostcode if it exists and postalcode if it doesn't)