I'm trying to import a Mandarin corpus (GlobalPhone Mandarin, aligned with the GP dict). Calling encode_tone_to_syllables(regex = "[1-5]$", clean_phone_label = True) produces the following error:
File "/home/linguistics/mlipar1/PolyglotDB/polyglotdb/corpus/syllabic.py", line 583, in encode_tone_to_syllables
self.enrich_syllables(enrich_dict)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/home/linguistics/mlipar1/PolyglotDB/polyglotdb/corpus/syllabic.py", line 492, in enrich_syllables
type_data = {k: type(v) for k, v in next(iter(syllable_data.values())).items()}
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
I think the problem has something to do with encode_tone_to_syllables() calling remove_pattern(regex) between creating enrich_dict and calling self.enrich_syllables(enrich_dict). But remove_pattern() itself re-runs self.encode_syllables("maxonset"), which can in principle cause issues---I see _generate_tone_enrichment() tries to anticipate issues due to the changing phone labels, but something seems to be going wrong.
More generally, is there a way we could simplify the code here/eliminate this extra encode_syllables() call? It adds a good amount of time to the import, but more importantly, it's forcing maxonset, which may not be what the user originally did (though in my case it was, so this isn;t where the issue is coming from).
I can share some example files (and my script) on Slack for reproducibility.
I'm trying to import a Mandarin corpus (GlobalPhone Mandarin, aligned with the GP dict). Calling
encode_tone_to_syllables(regex = "[1-5]$", clean_phone_label = True)produces the following error:I think the problem has something to do with
encode_tone_to_syllables()callingremove_pattern(regex)between creatingenrich_dictand callingself.enrich_syllables(enrich_dict). Butremove_pattern()itself re-runsself.encode_syllables("maxonset"), which can in principle cause issues---I see_generate_tone_enrichment()tries to anticipate issues due to the changing phone labels, but something seems to be going wrong.More generally, is there a way we could simplify the code here/eliminate this extra
encode_syllables()call? It adds a good amount of time to the import, but more importantly, it's forcingmaxonset, which may not be what the user originally did (though in my case it was, so this isn;t where the issue is coming from).I can share some example files (and my script) on Slack for reproducibility.