GeoNames Home | Postal Codes | Download / Webservice | About 

GeoNames Forum
  [Search] Search   [Recent Topics] Recent Topics   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
Errors in asciiname entries: empty or non-ASCII  XML
Forum Index -> General
Author Message
crontab



Joined: 18/12/2010 15:46:04
Messages: 2
Offline

I'm trying to import all class 'P' entries in allCountries.txt to a MySQL database. To optimize my search indexes I decided to change the charset setting for the asciiname column to use the 'ascii' charset. Looks like overall the entries conform to 7-bit ASCII with some exceptions. The extract below is a result of a query that selects empty asciiname entries plus those that contain question marks (?) -- presumably Unicode chars that can't be represented in 7-bit ASCII. There are total of 305 of them and they clearly can be fixed.

Please note that, again, these are only from the class 'P' subset. Naturally, there may be many more in non-P features.

The selected columns are: geonameid, name, asciiname, tab-delimited.

Code:
 7501451	مینو	????
 146970	Xankǝndi	Xank?ndi
 147622	Lǝnkǝran	L?nk?ran
 6618546	Anta Q´hawa	
 6618571	Luyuluyu Ck´asa	
 6618574	K´allpa	
 7535694	Métabetchouan–Lac-à-la-Croix	Metabetchouan?Lac-a-la-Croix
 7397284	Da?tun	Da?tun
 7397780	Zhong?tun	Zhong?tun
 7399112	Guo?zi	Guo?zi
 7402101	?zigou	?zigou
 7402385	??wo	??wo
 7506579	雄鸡埭	???
 6545245	Saint John´s woods	Saint John?s woods
 6545246	St. John´s wood	St. John?s wood
 7530607	ღვაშიგვგრდი	???????????
 6735908	Jo’ong	
 7290222	Periyapattinam (Tamil:பெரிய‌ப‌ட்டின‌ம்)	Periyapattinam (Tamil:periya?pa?ttina?m)
 98576	آشورن	
 6647876	Ḩayy al Bayḑāʼ	
 6647881	Ḩayy al Janāʼin	
 6647896	Ḩayy al Muʻtaşam	
 6647926	Qaryat Khaḑīr ʻAbbās	
 7501320	Qaryat as Saʻdūn	Qaryat as Sa?dun
 7501321	Qaryat ʻAbd Allāh as Sarāy	Qaryat ?Abd Allah as Saray
 7501323	Ibrāhīm ʻAţīyyah	Ibrahim ?At?iyyah
 7501326	Ḩamīd Ḩasan	H?amid H?asan
 7501328	Qaryat ʻAlīy Suhayl	Qaryat ?Aliy Suhayl
 7501463	Qaryat al Muʼāḑīd	Qaryat al Mu?ad?id
 7501468	ʻUbayd Dafār	?Ubayd Dafar
 7501476	Qaryat Ālbū ʻĪsá	Qaryat Albu ?Isa
 7501477	Aḩmad Daḩām	Ah?mad Dah?am
 7501505	Qaryat Salmān aḑ Ḑārī	Qaryat Salman ad? D?ari
 7501558	Qaryat Kāz̧im ʻIdhāb	Qaryat Kazim ?Idhab
 43864	صادق آباد	
 118876	Raʼīs	
 120523	Qal‘eh Ḩasan	
 123110	Mūʼīneh	
 132988	Gomānj-e Pāʼīn	
 139798	Būʼīn-e Soflá	
 404980	Qal‘eh Solţān Bājī	
 6648088	Sho‘eyţeh	
 6648638	Khānehhā-ye Sāzemānī-ye Şanāye‘-e Fūlād	
 6677119	Tāzeh Qalˈeh	
 6677476	‘Alī Şafar	
 6677687	Kalāteh-ye Ḩājjī ‘Abdol	
 6685385	‘Abdol Ḩoseyn Kandī	
 6686968	Shahrak-e Maskūnī-ye Valī-ye ‘Aşr	
 6687741	Ḩamūdī-ye Saʻdūn	
 6709097	Gerūsheh-ye ‘Olyā	
 6709100	Qal‘eh Tūt-e ‘Olyā	
 6709112	Banjū-ye ‘Olyā	
 6709471	‘Aşrābād-e Kohneh	
 6711433	La‘lābād-e Ḩoseyn Qolīkhānī	
 6711465	Ḩājat ‘Alī	
 6719833	Morād ʽAlī	
 6719926	Shahrak-e Valīye‘aşr	
 6720817	Shahrak-e Beʻs̄at	
 6727552	Shahrak-e Şan‘atī-ye Jānbāzān	
 6731125	Ḩājjī Khān ‘Alī	
 6731126	Ḩājjī ‘Avaẕ	
 6731137	Qeshlāq-e Ḩājj ‘Īsá (2)	
 6731138	Qeshlāq-e Ḩājj ‘Īsá (1)	
 6758151	‘Alī Maḩalleh	
 6758159	Sa‘ādat Maḩalleh	
 6776677	Noşratābād-e ‘Olyá	
 6776710	‘Ālamābād-e Soflá	
 7011502	قلعه سنگ	
 7347231	‘Avaẕ Ḩājjī	`Avaz H?ajji
 7351313	Ḩoseynābād	H?oseynabad
 7351314	Moḩammadābād	Moh?ammadabad
 7380791	Dashtak-e Vosţá	Dashtak-e Vost?a
 7388749	Ḩodyyā	H?odyya
 7389077	Maḩal-e Eḩdās̄-e Shahrak-e Qodūs	Mah?al-e Eh?das-e Shahrak-e Qodus
 7389078	Chūlak-e Şādeqābād	Chulak-e S?adeqabad
 7389111	Shahrak-e Shahīd Ḩamīdī	Shahrak-e Shahid H?amidi
 7406972	Mīrfattāḩ	Mirfattah?
 7407007	Zanganeh-e Soflá	Zanganeh-e Sofla?
 7407011	Magasān-e Soflá	Magasan-e Sofla?
 7407020	Shahrak-e Shahīd Moḩammad-e Borūjerdī	Shahrak-e Shahid Moh?ammad-e Borujerdi
 7414671	Tall-e Vostá	Tall-e Vosta?
 7416567	Faraḩānī-ye Moqaddam	Farah?ani-ye Moqaddam
 7416607	Arbīḩāt	Arbih?at
 7416609	Ālbū Mūsá	Albu Musa?
 7416611	Dowraqī-ye Ālbū Mūsá	Dowraqi-ye Albu Musa?
 7416622	Şabbāḩīyeh	S?abbah?iyeh
 7416623	Şabbāḩīyeh	S?abbah?iyeh
 7416624	Şafā’ī	S?afa'i
 7416642	Nahr-e Raḩmeh	Nahr-e Rah?meh
 7416643	Mendovān-e ‘Aţīyeh	Mendovan-e `At?iyeh
 7425535	Vasaţ Maḩalleh-ye Kharā Rūd	Vasat Mah?alleh-ye Khara Rud
 7425536	Bālā Maḩalleh-ye Kharā Rūd	Bala Mah?alleh-ye Khara Rud
 7457210	Moḩammad Ḩoseynābād	Moh?ammad H?oseynabad
 7457216	Bāgh-e Movaḩed	Bagh-e Movah?ed
 7490317	Nehẕatābād	Nehz?atabad
 7490374	Ẕarghāmābād	Z?arghamabad
 7529548	Sar Por-e Nūr Moḩammadī	Sar Por-e Nur Moh?ammadi
 7529871	Deh-e ‘Avaẕ	Deh-e `Avaz?
 7596964	΄Alī Kandī	?Ali Kandi
 7616389	Faẕlābād	Faz?labad
 7625081	Ḩasanābād	H?asanabad
 7625486	Ţālesh Maḩalleh-ye Ketālem	T?alesh Mahalleh-ye Ketalem
 7625512	Ţālesh Sarā	T?alesh Sara
 6281260	An­gol	
 6281261	Ansŏng­gol	
 6281270	Ching­ni	
 6281279	Chungsang­ni	
 6281286	Ipsŏng­ni	
 6281287	Kaltae­dong	
 6281294	Kogae­gol	
 6281303	Nae­dong	
 6281313	Sŏng­dong	
 6281322	Pŏltŏk­kol	
 6281323	Pugyŏl­li	
 6281327	Saek­kol	
 6281340	Tongdonggŏmŭn­gol	
 6281341	Ŭmyang­dong	
 6281344	Yongam­ni	
 6281347	Yongdŏng­ni	
 6302777	Chŭngsal­li	
 6305267	Segok­tong	
 6305268	Tomak­tong	
 6305269	Paegam­dong	
 6305271	Mangdan­dong	
 6305277	Chosin­dong	
 6363922	Pŏlp´an	
 6363967	Sangdaech´ang	
 6364142	Ugŏch´on	
 6364260	T´apkol	
 6799199	T’ǒ-gol	
 6799218	Chech’ǒng-mal	
 6802104	Korant’ǒ	
 2378335	El ‛Erg	
 7370519	Goth Hāji Sohrāb Khān Jakhrāده	Goth Haji Sohrab Khan Jakhra??
 7389281	Sar Būlāk	Sar B?l?k
 7389430	Lālo Qila	L?lo Qila
 7389541	Nawāb Kārez	Naw?b K?rez
 7418436	Vazīri Gund	Vazi?ri Gund
 474109	Vesёloye	
 583509	Ak”yar	Ak?yar
 797381	Ozёrnoye	
 816188	Kovalёvshchina	
 816729	Shmelёvo	
 817448	Berёzovka	
 818241	Zelёnyy Klin	
 818401	Zhёltaya Akatsiya	
 818778	Belёvaya	
 819321	Vorob’yёvka	
 819705	Morёvo	
 819709	Zelёnyy Gay	
 819756	Korolёvka	
 7411316	Ponоmarevo	Pon?marevo
 7434419	Grachёvka	Grach?vka
 7434454	Vesёlyy	Ves?lyy
 7434526	Krasnyy Posёlok	Krasnyy Pos?lok
 7434539	Kovalёvka	Koval?vka
 7434541	Semёnovodcheskiy	Sem?novodcheskiy
 7434576	Vishnёvka	Vishn?vka
 7436295	Belozёrnyy	Beloz?rnyy
 7436344	Zelёnaya Balka	Zel?naya Balka
 7436353	Opёnki	Op?nki
 7436411	Klёny	Kl?ny
 7437892	Bakharёvskiy	Bakhar?vskiy
 7442300	Malyye Ozёra	Malyye Oz?ra
 7450090	Berёzovka	Ber?zovka
 7450101	Yerёmin	Yer?min
 7450109	Kulёvo	Kul?vo
 7450135	Zelёnyy Gay	Zel?nyy Gay
 7450136	Vishnёvskiy	Vishn?vskiy
 7450187	Alёshkin	Al?shkin
 7451366	Berёzovaya	Ber?zovaya
 7451382	Fёdorovskiy	F?dorovskiy
 7451435	Zelёnyy Rog	Zel?nyy Rog
 7451437	Zelёnaya Roshcha	Zel?naya Roshcha
 7451438	Vesёlyy	Ves?lyy
 7451439	Zelёnyy Kust	Zel?nyy Kust
 7454571	Zaozёr’ye	Zaoz?r'ye
 7454763	Berёzovskaya	Ber?zovskaya
 7454813	Zaozёr’ye	Zaoz?r'ye
 7493616	Konёvka	Kon?vka
 7493620	Khmelёvitsa	Khmel?vitsa
 7493634	Konёvka	Kon?vka
 7496459	Berёziny	Ber?ziny
 7496482	Gremyachёvskaya	Gremyach?vskaya
 7496494	Ponomarёvskaya	Ponomar?vskaya
 7496555	Vesёlaya Griva	Ves?laya Griva
 7590869	Zalivnoе	Zalivno?
 7595007	Sтаroye Baltachëvo	S??roye Baltachevo
 7604674	Trёkhrech’ye	Tr?khrech'ye
 7604741	Kotёnovskiy	Kot?novskiy
 7353780	Al Ḩabīl	Al H?abil
 7357833	Al Faţīḩah	Al Fat?ih?ah
 7366853	Qarn Aḩad Yūbah	Qarn Ah?ad Yubah
 7367673	Al Makhlūţah	Al Makhlut?ah
 7370930	Āl Ḩalāmī	Al H?alami
 7371000	Āl Ḩadīlah	Al H?adilah
 7439930	At Tibāḩ	At Tibah?
 6943439	كاف الجاع	
 6855641	Dalakéna Ⅱ	
 6690124	สี่แยก อสมท	
 6691800	บ้านตลาดใหม่	
 7503210	Gϋzelyalı	G?zelyali
 73282	Maḩallat al ̒Uways	
 406587	Al Ḩamrā̛	
 406689	Āl ‘Ammār	
 406933	Az̧ Zā̧li‘	
 407070	Qaryat as Sabī ̔	
 7341021	Al Maḩaţţah	Al Mahat?t?ah
 7341723	Al Mahādimah Maḩāll aţ Ţaraf	Al Mahadimah Mahall at T?araf
 7341827	Ḩaḑan a̧s Şabl	Hadan a?s Sabl
 7342177	Şāfiyat Ţāmish	Safiyat T?amish
 7342477	Hi̧şn Bayt Na‘am	Hi?sn Bayt Na`am
 7345331	Khuḑā‘ah	Khud?a`ah
 7345333	Āl Jālī	A?l Jali
 7345339	Ar Rajaḩ	Ar Rajah?
 7346043	Al Kadā’	Al Kada?'
 7346048	Thāhir Dijāj	Thahir Dija?j
 7346070	Jāḑi‘ Muşabbiḩ	Jad?i` Musabbih
 7346081	Aş Şarw	As? Sarw
 7346094	Ḩilf Banī al Qamm	H?ilf Bani al Qamm
 7346098	Dhī al Bawāqil	Dhi al Bawa?qil
 7346108	Ţaybān	Tayba?n
 7346114	Ahl Şa‘dah	Ahl S?a`dah
 7346115	Şarw Ḩayyān	Sarw Hayya?n
 7346140	Gharbī Sha‘bān	Gharbi Sha`ba?n
 7346146	Suwār	Suwa?r
 7346177	Uqrun Āl Ja‘dān	Uqrun A?l Ja`da?n
 7346185	Duwāḩ	Duwah?
 7346219	Āl Abū Ghubārah	A?l Abu Ghubarah
 7346221	Alt ‘Ayshān	Alt `Aysha?n
 7346222	Maşḩan	Mas?h?an
 7346247	Ibn Lakwān	Ibn Lakwa?n
 7346266	Alt ‘Āmir	Alt `A?mir
 7346313	Adh Dhārī	Adh Dha?ri?
 7346316	Al ‘Awārij	Al `Awa?rij
 7346318	Ad Dumāh	Ad Duma?h
 7346319	Aţ Ţāh	At Ta?h
 7346326	Tī Yarkham	Ti? Yarkham
 7346334	‘Āsī al Qarn	`A?si al Qarn
 7346335	Aş Şa‘rūf	As Sa`ru?f
 7346344	Adh Dhirā‘	Adh Dhira?`
 7346346	Ḩablān	Habla?n
 7346347	Al Baqāmah	Al Baqa?mah
 7346379	Al Ḩaḑīrah	Al Hadi?rah
 7346400	Qullat al Ḩamrī	Qullat al Hamri?
 7346401	Şāyid	S?ayid
 7346416	Al Maşna‘ah	Al Mas?na`ah
 7346458	Āl aţ Ţawq	Al at T?awq
 7346496	Banī Sa‘d	Bani? Sa`d
 7346509	Adh Dhirā‘	Adh Dhira?`
 7346516	Alt al Muţayrī	Alt al Mut?ayri
 7346537	Tikinnāyah	Tikinna?yah
 7346539	An Nab‘ī	An Nab`i?
 7346548	Bilād Qudam	Bila?d Qudam
 7346553	Aqārin	Aqa?rin
 7346560	Ad Dārah	Ad Da?rah
 7346676	Alt ‘Uţayf	Alt `Ut?ayf
 7354857	Raha̧bah	Raha?bah
 7355347	Al Mudayrīş	Al Mudayris?
 7356955	‘Arḑ al A‘wir	`Ard? al A`wir
 7357208	Z̧ābi ̓	Zabi ?
 7358714	̒̒Adayrah	??Adayrah
 7359992	Aţ Ţabaqahb ̒Ard al Jabūb	At? T?abaqahb ?Ard al Jabub
 7360133	Nuqţat Banī Shihāb	Nuqt?at Bani Shihab
 7360954	Bayḩān al ‘Uly̧ā	Bayhan al `Uly?a
 7361264	Aş Şayyāţ	As Sayyat?
 7361723	Bayt ash Shaḩţirah	Bayt ash Shaht?irah
 7361738	Quţayn al Qashāmī	Qut?ayn al Qashami
 7361917	Bayt al Khubāţ	Bayt al Khubat?
 7361930	Bayt Ḩūmī	Bayt H?umi
 7362051	Ash Shijān	Ash Shija?n
 7363127	Al Ḩamrā’	Al H?amra'
 7363141	Al Lāţī	Al Lat?i
 7363142	Al Ma‘faţ	Al Ma`fat?
 7363143	Al Madārah	Al Mada?rah
 7363187	Mirāḩ Quzzān	Mirah? Quzzan
 7364429	̧Jabal aş Şafrā’	?Jabal as Safra'
 7364440	Maḩaţţat al ‘Ubaydī	Mahat?t?at al `Ubaydi
 7364699	Qaryat al ‘Aţf	Qaryat al `At?f
 7364724	Al Wāsiţah	Al Wasit?ah
 7364784	Al ̒Aqabah	Al ?Aqabah
 7364898	Al Ḩaţab	Al Hat?ab
 7365070	Maḩall Bulghat Fīd	Mahall Bulghat Fi?d
 7365266	Maḩall al Ḩisņ	Mahall al Hisn?
 7365312	Adh Dhībīnīyah	Adh Dhibi?niyah
 7365334	Al Kadḩīyah	Al Kadh?iyah
 7365381	Dayr Bāshā	Dayr Basha?
 7365392	Ḑubāyah	Duba?yah
 7365464	Maḩall al ‘Aḑābīyah	Mahall al `Ad?abiyah
 7365503	Yaḩyá aş Şaghīr	Yahya as? Saghir
 7365772	‘Abs Thawāb	`Abs Thawa?b
 7365915	Al Quţa‘	Al Qut?a`
 7366321	Maḩall ad Dimnah aţ Ţawīlah	Mahall ad Dimnah at T?awilah
 7366339	Dayr Zābiţ	Dayr Zabit?
 7366368	Maḩall al Qurayţī	Mahall al Qurayt?i
 7366438	Al Muqdhūlah	Al Muqdhu?lah
 7366488	Al Maţ‘an	Al Mat?`an
 7366504	Maḩall Banī Āmir al Ḩawātim	Mahall Bani Amir al H?awa?tim
 7366565	Al Kassās	Al Kassa?s
 7366591	Bayt Ţaḩāmah	Bayt T?ahamah
 7366797	Az̧ Z̧ahār	Az Zaha?r
 7366802	Abū al Khadārīsh	Abu al Khada?rish
 7366833	Al Balāyiţ	Al Balayit?
 7366864	̒̒Aqabāt	??Aqabat
 7367271	Ḩişn aḑ Ḑabyatayn	His?n ad Dabyatayn
 
marc



Joined: 08/12/2005 07:39:47
Messages: 4501
Offline

Thanks, I will run a check and set the asciiname to null for the problematic entries.

Cheers

Marc

[WWW]
crontab



Joined: 18/12/2010 15:46:04
Messages: 2
Offline

Hi Marc

(BTW notifications from the forum end up in Spam at least on GMail, just to let you know...)

A quick glance at the non-ASCII names shows that: in many of these names the non-ASCII character could have been omitted, e.g.

Moḩammad Ḩoseynābād -> Moh?ammad H?oseynabad -> Mohammad Hoseynabad

Also, there are some Russian names where the letter "ё" is used (pronounced "yo")

Vishnёvka -> Vishn?vka

I'm not sure what this letter is translated to in Geographical names traditionally, could be either "yo" or "e". Again, can be replaced easily in either case.
marc



Joined: 08/12/2005 07:39:47
Messages: 4501
Offline

I have run an update:
7566171 rows checked, 621 updated, 348 failed.

If you want the 348 failed ones to be not null you are welcome to update them.

Best

Marc

[WWW]
Pit



Joined: 08/06/2011 09:02:59
Messages: 2
Offline

Edit: forget about it, wanted to commit some fixes but I just noticed that my ascii converter messed up
 
Forum Index -> General
Go to:   
Powered by JForum 2.1.5 © JForum Team