<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
	<channel>
		<title><![CDATA[Latest posts for the topic "Import GeoNames dump into Postgres"]]></title>
		<link>http://forum.geonames.org/gforum/posts/list/6.page</link>
		<description><![CDATA[Latest messages posted in the topic "Import GeoNames dump into Postgres"]]></description>
		<generator>JForum - http://www.jforum.net</generator>
			<item>
				<title>Import GeoNames dump into Postgres</title>
				<description><![CDATA[ I use simple psql to import the dump into postgres. First I create a file create.sql :

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>
create table geoname &#40;
        geonameid       int,
        name            varchar&#40;200&#41;,
        asciiname        varchar&#40;200&#41;,
        alternatenames  varchar&#40;4000&#41;,
        latitude        float,
        longitude       float,
        fclass  char&#40;1&#41;,
        fcode   varchar&#40;10&#41;,
        country varchar&#40;2&#41;,
        cc2 varchar&#40;60&#41;,
        admin1  varchar&#40;20&#41;,
        admin2  varchar&#40;80&#41;,
        admin3  varchar&#40;20&#41;,
        admin4  varchar&#40;20&#41;,
        population      bigint,
        elevation       int,
        gtopo30         int,
        timezone varchar&#40;40&#41;,
        moddate         date
&#41;;

create table alternatename &#40;
        alternatenameId  int,
        geonameid          int,
        isoLanguage        varchar&#40;7&#41;,
        alternateName     varchar&#40;200&#41;,
        isPreferredName      boolean,
        isShortName    boolean
&#41;;

Note: the countryInfo file has changed since this posting was written.


CREATE TABLE "countryinfo" &#40;
     iso_alpha2 char&#40;2&#41;,
     iso_alpha3 char&#40;3&#41;,
     iso_numeric integer,
     fips_code character varying&#40;3&#41;,
     name character varying&#40;200&#41;,
     capital character varying&#40;200&#41;,
     areainsqkm double precision,
     population integer,
     continent char&#40;2&#41;,
     languages character varying&#40;200&#41;,
     currency char&#40;3&#41;,
     geonameId int
&#41;;
</pre>
		</div>

Then I create the tables with 'psql dabase_name < create.sql'

in the psql command line I enter (copy/paste) :

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>
copy geoname &#40;geonameid,name,asciiname,alternatenames,latitude,longitude,fclass,fcode,country,cc2, admin1,admin2,admin3,admin4,population,elevation,gtopo30,timezone,moddate&#41; from 'allCountries.txt' null as '';
copy alternatename  &#40;alternatenameid,geonameid,isoLanguage,alternateName,isPreferredName,isShortName&#41; from 'alternateNames.txt' null as '';
copy countryInfo  &#40;iso_alpha2,iso_alpha3,iso_numeric,fips_code,name,capital,areaInSqKm,population,continent,languages,currency,geonameId&#41; from 'countryInfo.txt' null as '';
</pre>
		</div>

 Edit 2006.08.07 : added cc2
 Edit 2006.08.08 : added timezone
 Edit 2007.03.24 : added admin2
 Edit 2007.04.01 : updated table alternatename, added isPreferredName
 Edit 2007.05.26 : update table alternateName, added isShortName
 Edit 2007.07.25 : add admin3 and admin4
 Edit 2007.07.30 : update table alternateName change isoLanguage from char(4) to char(7) to allow the pseudo code fr_1793
 Edit 2007.08.23 : update countryInfo add currency and geonameId
 Edit 2007.09.19 : remove quotes from countryInfo create table
 Edit 2009.01.10 : http://forum.geonames.org/gforum/posts/list/1208.page
 Edit 2009.04.13 : alter column population to bigint (from int) to allow for the population of continents.]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#174</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#174</link>
				<pubDate><![CDATA[Mon, 15 May 2006 18:10:52]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import geonames dump into Postgres</title>
				<description><![CDATA[ Marc,
 
Can you update the helpful instructions you've provided for creating the alternatenames table and the copy command for loading the alternate names file into Postgres?  

alternateNameId   : the id of this alternate name, int
geonameid         : geonameId referring to id in table 'geoname', int
isolanguage       : iso 693 language code 2- or 3-characters, varchar(3)
alternate name    : alternate name or name variant, varchar(200)
isOfficialName    : '1', if this alternate name is an official/preferred name

Thank you.

Dan]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#1802</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#1802</link>
				<pubDate><![CDATA[Sun, 1 Apr 2007 05:15:02]]> GMT</pubDate>
				<author><![CDATA[ dpc_map]]></author>
			</item>
			<item>
				<title>Re:Import geonames dump into Postgres</title>
				<description><![CDATA[ Dan

Thanks. 
I have also updated the documentation for the column isoLanguage. We are now including iata/icao codes for airports :
isolanguage       : iso 693 language code 2- or 3-characters; 4-characters 'iata' or 'icao' for airport codes, varchar(4)

Cheers

Marc]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#1803</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#1803</link>
				<pubDate><![CDATA[Sun, 1 Apr 2007 09:05:18]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import geonames dump into Postgres</title>
				<description><![CDATA[ A 4 character-long code can also be "post", for postal codes]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#1935</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#1935</link>
				<pubDate><![CDATA[Sun, 22 Apr 2007 14:15:49]]> GMT</pubDate>
				<author><![CDATA[ samokk]]></author>
			</item>
			<item>
				<title>Re:Import geonames dump into Postgres</title>
				<description><![CDATA[ Thanks, I have updated the documentation.

Marc]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#1940</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#1940</link>
				<pubDate><![CDATA[Mon, 23 Apr 2007 07:32:06]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Hello,

thanks a lot for your documentation.
I've found some issues, you can possibly fix in your documentation

<b>countryInfo.txt</b> contains two more fields:
currency =>  character(3),
geonameid =>  integer NOT NULL

No Primary Keys are defined, i've used the following ones:

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>ALTER TABLE ONLY alternatename
    ADD CONSTRAINT pk_alternatenameid PRIMARY KEY &#40;alternatenameid&#41;;

ALTER TABLE ONLY geoname
    ADD CONSTRAINT pk_geonameid PRIMARY KEY &#40;geonameid&#41;;

ALTER TABLE ONLY countryinfo
    ADD CONSTRAINT pk_iso_alpha2 PRIMARY KEY &#40;iso_alpha2&#41;;</pre>
		</div>

I wasn't able to define Foreign-Keys for the <b>geonameid</b> on the tables <b>alternatename</b> and <b>countryinfo</b>. The data seems to be inconsistent on some tupels. So I decided to use the following indices:

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>
CREATE INDEX index_countryinfo_geonameid ON countryinfo USING hash &#40;geonameid&#41;;

CREATE INDEX index_alternatename_geonameid ON alternatename USING hash &#40;geonameid&#41;;</pre>
		</div>

Maybe you know a better solution?

Greetings


Andy]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2701</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2701</link>
				<pubDate><![CDATA[Thu, 23 Aug 2007 17:57:53]]> GMT</pubDate>
				<author><![CDATA[ pinhead84]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Thanks, I have updated the countryInfo doc.

You could help us sort out the missing or wrong geonameIds in the countryInfo file.

Cheers

Marc]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2705</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2705</link>
				<pubDate><![CDATA[Thu, 23 Aug 2007 18:43:24]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Hello Marc,

thanks for your quick answer. 

<p></p>

		<cite>marc wrote:</cite><br>
		<blockquote>You could help us sort out the missing or wrong geonameIds in the countryInfo file.&nbsp;
		</blockquote>

One simple problem on <b>countryinfo</b>-Table:
<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>SELECT * FROM countryinfo WHERE geonameid=0;</pre>
		</div>

Brings up 17 tupels with the following iso_alpha2:

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>
AQ
AS
AX
BV
FO
GF
GU
HM
MP
MQ
NC
PM
RE
SJ
UM
VI
WF</pre>
		</div>

If these countries aren't available on the <b>geoname</b>-table, I would suggest to create a dummy-entry with "geonameid=0".

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>INSERT INTO geoname &#40;geonameid&#41; VALUES &#40;0&#41;;</pre>
		</div>

After this modification, the foreign-key on <b>countryinfo</b> is set up correctly:

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>ALTER TABLE ONLY countryinfo
    ADD CONSTRAINT fk_geonameid FOREIGN KEY &#40;geonameid&#41; REFERENCES geoname&#40;geonameid&#41;;</pre>
		</div>

On the <b>alternatename</b>-table the problem can't be fixed so easy. I'll try to find out the corrupted tupels and will tell them to you on this thread.

Greetz


Andy]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2707</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2707</link>
				<pubDate><![CDATA[Thu, 23 Aug 2007 19:10:36]]> GMT</pubDate>
				<author><![CDATA[ pinhead84]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Hi Andy

These countries are available, only the geonameId is missing in the countryInfo. You could add the geonameId to your list above and I will add it then to the countryInfo.txt file.

Marc

PS: It is related to another thread : http://forum.geonames.org/gforum/posts/list/570.page]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2711</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2711</link>
				<pubDate><![CDATA[Thu, 23 Aug 2007 20:11:20]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Hi Marc,

<p></p>

		<cite>marc wrote:</cite><br>
		<blockquote>These countries are available, only the geonameId is missing in the countryInfo. You could add the geonameId to your list above and I will add it then to the countryInfo.txt file.&nbsp;
		</blockquote>

I tried to check the broken relations between <b>countryInfo</b> and <b>geoname</b> tables, but had some problems finding out the corresponding <b>geoname</b>-tupels. I would suggest to use the following relations:

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>
AS =&gt; geonameid=5880801; fclass=A; fcode=ADMD; name=American Samoa, Territory of
BV =&gt; geonameid=3371123; fclass=A; fcode=PCLD; name=Bouvet Island
FO =&gt; geonameid=2622320; fclass=A; fcode=ADM1; name=Faroe Islands
GF =&gt; geonameid=3381670; fclass=A; fcode=ADM1; name=Department of Guiana
GU =&gt; geonameid=4043988; fclass=A; fcode=ADMD; name=Territory of Guam
MP =&gt; geonameid=4041467; fclass=A; fcode=ADMD; name=Northern Islands
MQ =&gt; geonameid=3570311; fclass=A; fcode=ADM1; name=Department of Martinique
NC =&gt; geonameid=2139685; fclass=A; fcode=ADM1; name=Territory of New Caledonia and Dependencies
PM =&gt; geonameid=3424932; fclass=A; fcode=ADM1; name=Territorial Collectivity of Saint Pierre and Miquelon
RE =&gt; geonameid=935317;  fclass=A; fcode=ADM1; name=Department of Reunion
SJ =&gt; geonameid=607072;  fclass=A; fcode=ADM1; name=Svalbard
UM =&gt; geonameid=5854968; fclass=A; fcode=ADMD; name=United States Minor Outlying Islands
VI =&gt; geonameid=4796775; fclass=A; fcode=ADMD; name=Virgin Islands of the United States
WF =&gt; geonameid=4034749; fclass=A; fcode=ADM1; name=Territory of the Wallis and Futuna Islands</pre>
		</div>

Because I'm relative new to the DB-structure of GeoNames, I've added the corresponding fclass and fcode fields to my listing. Is it correct, that "fclass=A" is a condition for this relations? - All the other relations between <b>countryInfo</b> and <b>geoname</b> (e.g. Germany) always have "fclass=A". Please correct me, if I misunderstood something. ;)

Assuming that I'm right, the following <b>countryinfo</b>-entries can not be related correctly to the <b>geoname</b>-table:

<span class="genmed"><b>Code:</b></span><br>
		<div style="overflow: auto; width: 100%;">
		<pre>
AQ =&gt; no entry in geoname-table &#40;WHERE country='AQ' AND fclass='A'&#41;
AX =&gt; no entry in geoname-table &#40;WHERE country='AX'&#41;
HM =&gt; no entry in geoname-table &#40;WHERE country='HM'&#41;</pre>
		</div>

These entries still relate to geonameid=0, so I still need the dummy-entry in the <b>geoname</b>-table to ensure a valid Foreign-Key-Constraint. Is it possible to create the corresponding entries in the <b>geoname</b>-table?

<p></p>

		<cite>andy wrote:</cite><br>
		<blockquote>On the <b>alternatename</b>-table the problem can't be fixed so easy. I'll try to find out the corrupted tupels and will tell them to you on this thread.&nbsp;
		</blockquote>

I've written a small Java-tool to check the broken relations on the <b>alternatename</b>-table. It found 245 geonameid's, that don't exist in the <b>geoname</b>-table (anymore?). I attach the validation-result to this posting. 

If you're interested, I would send you the tool including (GPL'ed) source-code.


Greetz


Andy]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2715</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2715</link>
				<pubDate><![CDATA[Fri, 24 Aug 2007 16:26:17]]> GMT</pubDate>
				<author><![CDATA[ pinhead84]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Hi there,

I am trying to fill an empty pgsql database based on the instructions here. I basically want to create a database which I can use to match populated places with in order to geocode them. 
I am getting an error when trying to copy the countryInfo.txt file. It complains about the column "geonamei" of relation "countryinfo" not existing. The column is there though is empty. 
Can someone point me in the right direction?! I am quite new to postgresql/postgis but eager to get fully acquainted. 
Is there a sql dump of a full database by any chance?! That would make things quite easy.

Thanks ahead for any help.

cheers,
Mark de Blois]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2772</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2772</link>
				<pubDate><![CDATA[Tue, 11 Sep 2007 18:05:02]]> GMT</pubDate>
				<author><![CDATA[ markdeblois]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Hi Mark

Did you make sure it is all spelt correctly/consistently? (a column geonamei probably  does not exist in your database schema)

Regards

Marc]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2779</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2779</link>
				<pubDate><![CDATA[Wed, 12 Sep 2007 17:22:12]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ It seems that the allCountries.txt file probably has a few lines at the top which get in the way. As the file is over 700mb big I cannot seem to find a powerful enough editor to modify it. I tried the copy with a country file in which I removed the first few lines (including the columns) which worked well. But I hope there is a better way than importing them one by one  :-o 

I added a spatial field as well and was able to fill the geometry with the coordinates from the latitude longitude field. The country dataset loads up fine in QGIS, although the geonameid does seem to have become 0 (it has a value when I look at it through pg3admin?!). Generating the primary and foreign keys seems to cause some difficulties...but slowly moving ahead.

BTW, does anyone know of a (semi) automated way of updating the database?! 

Thanks!

Mark]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2781</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2781</link>
				<pubDate><![CDATA[Wed, 12 Sep 2007 18:15:41]]> GMT</pubDate>
				<author><![CDATA[ markdeblois]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Just had the same problem with the countryInfo upload claiming that column geonameId doesn't exist. I'm completely new to postgres, but it appears to be a capitalization issue. The sample 'create.sql' file at the head of this thread quotes the field names for the countryInfo table, forcing the I in geonameId to uppercase. psql appears to lowercase field names typed in (I'm using 8.2.4 on debian testing), so it never matches the stored field name.

To fix, just remove the quotes round the field names before creating the table.

Graham 

]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2816</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2816</link>
				<pubDate><![CDATA[Wed, 19 Sep 2007 13:30:07]]> GMT</pubDate>
				<author><![CDATA[ marinheiro]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ Thanks Graham. I have removed the quotes.

Marc]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#2817</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#2817</link>
				<pubDate><![CDATA[Wed, 19 Sep 2007 13:45:08]]> GMT</pubDate>
				<author><![CDATA[ marc]]></author>
			</item>
			<item>
				<title>Re:Import GeoNames dump into Postgres</title>
				<description><![CDATA[ http://forum.geonames.org/gforum/posts/list/926.page does more automatically]]></description>
				<guid isPermaLink="true">http://forum.geonames.org/gforum/posts/list/67.page#5018</guid>
				<link>http://forum.geonames.org/gforum/posts/list/67.page#5018</link>
				<pubDate><![CDATA[Wed, 26 Nov 2008 10:23:50]]> GMT</pubDate>
				<author><![CDATA[ bwakkie]]></author>
			</item>
	</channel>
</rss>