can't import dictionaries into yomitan | Refold – 日本語 (JA) | Page 1

lunar badge Jan 16, 2024, 3:34 AM

#

I'm trying to setup yomitan on my desktop and I can't get half of the dictionaries I downloaded to import. When I try I get a message saying "error: unknown error." All the dictionaries I'm trying to use are straight from the MarvNC yomichan dictionaries GitHub page and I didn't have any issues using them on my laptop.

hearty shuttle Jan 16, 2024, 4:36 AM

#

any chance you're out of storage or something

#

otherwise try reinstalling yomitan

lunar badge Jan 16, 2024, 10:43 AM

#

No, I have plenty of storage but I can try reinstalling

lunar badge Jan 16, 2024, 3:17 PM

#

Actually when I was messing with it last night I tried using yomitan on Chrome and brave browser

#

And had the same dictionaries produce the same error while importing so I don't think reinstalling will work

hearty shuttle Jan 16, 2024, 7:55 PM

#

which dictionaries in specific?

#

did you download from the google drive folder at the top or from the direct links

#

for some of the dictionaries if you download them directly you need to unzip it first

#

for the google drive they should be good out of the gate

lunar badge Jan 16, 2024, 8:59 PM

#

hearty shuttle for the google drive they should be good out of the gate

I remember some were from the drive and then when those didn't work I went to the direct links

#

But I'll have to check which specifically when I get home

hearty shuttle Jan 16, 2024, 9:00 PM

#

uhhh make an issue on the yomitan github that's weird

#

specify os, browser, files, etc

lunar badge Jan 16, 2024, 9:51 PM

#

[Monolingual] 三省堂国語辞典　第八版 (Recommended), [Monolingual, Encyclopedia] Pixiv, [Freq] JPDB (Recommended), Freq.JPDB_2022-05-10T03_27_02.930Z, and Freq_CC100

#

were the ones I couldn't import

molten gyro Jan 16, 2024, 9:52 PM

#

Those are the same dictionaries peshti who had the same bug couldn't import:
#1194991266014969937 message

lunar badge Jan 16, 2024, 9:53 PM

#

huh so it's not just me

#

i don't really ever use github. how do i make an issue?

#

oh i need an account

molten gyro Jan 16, 2024, 10:02 PM

#

the only thing that comes to mind is that both [Freq] CC100 and [Freq] JPDB only have 1 "big" >10mb term_meta_bank_1.json
whereas other frequency dictionaries like the [Freq] BCCWJ-LUW are separated into 10+ small term_meta_bank_N.json files
feel free to ignore this remark if you believe its irrelevant to the current problem

lunar badge Jan 16, 2024, 10:05 PM

#

Sorry desktop is currently exploding

#

So this might be completely irrelevant

#

Alright it's alive again

molten gyro Jan 16, 2024, 10:11 PM

#

lunar badge Alright it's alive again

can you try editing the term_meta_bank_1 in your CC100?

#

only keep the words till 10k frequency

lunar badge Jan 16, 2024, 10:12 PM

#

I did get bccwj to import

molten gyro Jan 16, 2024, 10:12 PM

#

yes bcwwj has no problem

lunar badge Jan 16, 2024, 10:12 PM

#

molten gyro only keep the words till 10k frequency

How

molten gyro Jan 16, 2024, 10:13 PM

#

📎 message.txt

#

extract cc100.zip

#

and replace the content of the term meta bank by what i sent and save (i just kept the first few entires)

#

then in your cc100 folder, select the 2 json files, right click on one of them, send to compressed folder, then try to import to yomitan

lunar badge Jan 16, 2024, 10:15 PM

#

I mean like how do I open it

#

Notepad didn't go well

molten gyro Jan 16, 2024, 10:16 PM

#

there shouldnt be any problem with notepad?

#

what does it say

lunar badge Jan 16, 2024, 10:16 PM

#

It was just a brick wall of text

molten gyro Jan 16, 2024, 10:16 PM

#

yes its normal

#

ctrl + a, delete all

#

then paste what i sent

lunar badge Jan 16, 2024, 10:17 PM

#

Oh and just paste in what you sent

#

Alright

#

This is just to test the theory right?

#

I shouldn't actually use this dictionary after doing this

molten gyro Jan 16, 2024, 10:18 PM

#

ye

#

its just the first 10k words atm

#

import it to yomitan

#

we can always delete the dictionary its not a problem

lunar badge Jan 16, 2024, 10:21 PM

#

alright that worked

molten gyro Jan 16, 2024, 10:21 PM

#

nice

#

the cc100 appears correctly right?

#

when scanning

lunar badge Jan 16, 2024, 10:22 PM

#

in the little preview yeah

molten gyro Jan 16, 2024, 10:22 PM

#

ok lets try with more words

lunar badge Jan 16, 2024, 10:22 PM

#

it's showing yomu as 162

molten gyro Jan 16, 2024, 10:22 PM

#

📎 message.txt

#

up to 40k frequency

lunar badge Jan 16, 2024, 10:23 PM

#

seems to be working

molten gyro Jan 16, 2024, 10:24 PM

#

type 禿 and try to scan it

lunar badge Jan 16, 2024, 10:24 PM

#

btw the only freq dictionary i've used before was innocent corpus

#

do these work the other way around

#

as in lower number is more common for these?

molten gyro Jan 16, 2024, 10:24 PM

#

162 for yomu means 162th most common word in the language

#

small number = super common

lunar badge Jan 16, 2024, 10:25 PM

#

ok so that's the opposite of innocent corpus

#

umm i can't seem to scan text unless i'm doing it wrong

#

but i can scan subtitles from asbplayer

molten gyro Jan 16, 2024, 10:26 PM

#

do it in the yomichan dictionary

#

or type 禿 in google and hit enter

lunar badge Jan 16, 2024, 10:26 PM

#

oh wait nvm

#

yeah i did it in google

molten gyro Jan 16, 2024, 10:26 PM

#

just want to see if this word is indeed in the dictionary

#

what does cc100 say

lunar badge Jan 16, 2024, 10:26 PM

#

not showing up

molten gyro Jan 16, 2024, 10:27 PM

#

not that?

lunar badge Jan 16, 2024, 10:27 PM

#

same with bccwj

#

at least not for the hage reading which i assume is what you want

#

oh yeah that reading shows up

#

but idk if i've heard that before

#

hage i've heard a bunch

molten gyro Jan 16, 2024, 10:27 PM

#

idk that word either, its just the last word in the file i sent you xd

lunar badge Jan 16, 2024, 10:28 PM

#

lol

molten gyro Jan 16, 2024, 10:28 PM

#

to see if all words are covered

#

📎 message.txt

#

same thing

#

if nothing works you'll still have cc100 for the most 40k common words so thats still nice

lunar badge Jan 16, 2024, 10:29 PM

#

shows up in bccwj but not cc100

molten gyro Jan 16, 2024, 10:29 PM

#

send a screenshot plz

#

and you mean with the previous file i sent?

lunar badge Jan 16, 2024, 10:30 PM

#

#

i mean for nicaragua

molten gyro Jan 16, 2024, 10:30 PM

#

why would you scan nicaragua??

#

scan 禿

#

and send a screen

lunar badge Jan 16, 2024, 10:31 PM

#

oh i thought that last message was you telling me to scan nicaragua lol

#

so you want me to repeat what i already did with the json but with this text file now

#

and then try scanning hage again

#

give me a sec

molten gyro Jan 16, 2024, 10:32 PM

#

lunar badge so you want me to repeat what i already did with the json but with this text fil...

yes but did it work with the 2nd file i sent

molten gyro Jan 16, 2024, 10:32 PM

#

lunar badge oh yeah that reading shows up

when you did that

#

i didnt understand if cc100 worked or not

lunar badge Jan 16, 2024, 10:33 PM

#

haven't tried yet

#

misunderstood what you wanted earlier

molten gyro Jan 16, 2024, 10:34 PM

#

just scan 禿

#

and send a screenshot

lunar badge Jan 16, 2024, 10:36 PM

#

#

huh definition 2 is bizarrely specific

molten gyro Jan 16, 2024, 10:37 PM

#

did you import the cc100 dictionary

lunar badge Jan 16, 2024, 10:38 PM

#

yeah after editing the json file

molten gyro Jan 16, 2024, 10:38 PM

#

there was no error?

lunar badge Jan 16, 2024, 10:38 PM

#

i tried using the second text file you sent and it errored out

#

so i went back to the first text file so i could scan hage

molten gyro Jan 16, 2024, 10:38 PM

#

the first out of the 3 right

lunar badge Jan 16, 2024, 10:38 PM

#

yeah

molten gyro Jan 16, 2024, 10:39 PM

#

so the first one works but the second gives you "unknown error" while importing it?

lunar badge Jan 16, 2024, 10:39 PM

#

oh wait i didn't even see the second one

#

the third one errored out

#

and it was not an unknown error

#

it was something like some term wasn't recognized

#

let me try it again

molten gyro Jan 16, 2024, 10:39 PM

#

yes and send a screenshot of the error

lunar badge Jan 16, 2024, 10:42 PM

#

molten gyro Jan 16, 2024, 10:43 PM

#

oh thats my fault

molten gyro Jan 16, 2024, 10:43 PM

#

lunar badge

anyway try with this:
#1196658690216968332 message

lunar badge Jan 16, 2024, 10:43 PM

#

forgot a bracket or something

molten gyro Jan 16, 2024, 10:45 PM

#

i forgot i could have just sent you the zip folder

lunar badge Jan 16, 2024, 10:45 PM

#

lol oh well

#

it imported

molten gyro Jan 16, 2024, 10:45 PM

#

does cc100 work on yomu?

lunar badge Jan 16, 2024, 10:46 PM

#

and kaburo shows up with a frequency of 39327

#

on cc100

molten gyro Jan 16, 2024, 10:46 PM

#

good

#

📎 term_meta_bank_1.json

#

well lets continue with json files if it works

lunar badge Jan 16, 2024, 10:47 PM

#

just replace the current json with this one?

molten gyro Jan 16, 2024, 10:47 PM

#

yes

#

its the full one actually

#

but do delete and replace it like we did earlier

#

i was thinking it could be a compression problem

lunar badge Jan 16, 2024, 10:48 PM

#

got unknown error on that one

molten gyro Jan 16, 2024, 10:48 PM

#

so maybe the size is the problem

#

📎 message.txt

#

half of the full dictionary

#

up to 80k

#

or maybe the problem is some weird character somewhere in the dictionary it bugs on

#

@lunar badge

lunar badge Jan 16, 2024, 10:53 PM

#

it's importing

molten gyro Jan 16, 2024, 10:54 PM

#

scan 岩手山

#

should give you 80422

lunar badge Jan 16, 2024, 10:55 PM

#

#

didn't quite work

molten gyro Jan 16, 2024, 10:55 PM

#

wait it didnt work for me neither

#

we prob need a real entry in another dictionary

#

lemme try another word

#

@lunar badge try 立県

#

lunar badge Jan 16, 2024, 10:57 PM

#

80821

#

yeah

molten gyro Jan 16, 2024, 10:57 PM

#

send a screen please

#

we never know

lunar badge Jan 16, 2024, 10:58 PM

#

molten gyro Jan 16, 2024, 10:59 PM

#

gg

#

📎 message.txt

#

3/4th of the dictionary

#

up to 120k

lunar badge Jan 16, 2024, 11:01 PM

#

👀

#

btw for freq dictionaries you leave the priority as 0 right?

#

or does it not matter

molten gyro Jan 16, 2024, 11:04 PM

#

i have them set at 0

lunar badge Jan 16, 2024, 11:04 PM

#

it imported

molten gyro Jan 16, 2024, 11:05 PM

#

ok

lunar badge Jan 16, 2024, 11:05 PM

#

thanks for all the help

molten gyro Jan 16, 2024, 11:06 PM

#

scan 膝かけ

#

#

dw

#

im almost as clueless as you lol

lunar badge Jan 16, 2024, 11:06 PM

#

i can't believe i didn't recognize that one

molten gyro Jan 16, 2024, 11:07 PM

#

didnt do the core 1 million? smh

lunar badge Jan 16, 2024, 11:07 PM

#

lunar badge Jan 16, 2024, 11:07 PM

#

molten gyro didnt do the core 1 million? smh

i used that word to mine hiza

#

coincidentally

molten gyro Jan 16, 2024, 11:07 PM

#

well then the problem must be within the last 40k words of the dictionary

#

and thats only for the cc100

#

not the other dictionaries

lunar badge Jan 16, 2024, 11:08 PM

#

yeah

#

so you think it's a file size thing?

molten gyro Jan 16, 2024, 11:08 PM

#

this or theres a weird character it doesnt like

#

maybe some rare kanji or idk

#

but those are just random guesses i know nothing of yomitan's code

lunar badge Jan 16, 2024, 11:09 PM

#

also how is it that nobody told me i could yomichan words within yomichan

molten gyro Jan 16, 2024, 11:09 PM

#

you didnt know?

#

its a setting

#

number of child popups

lunar badge Jan 16, 2024, 11:09 PM

#

nah happened to see it just now while going through the settings

molten gyro Jan 16, 2024, 11:09 PM

#

alas

lunar badge Jan 16, 2024, 11:09 PM

#

but i asked about it a while back

#

the struggles of not reading

#

and depending on the kindness of strangers

molten gyro Jan 16, 2024, 11:13 PM

#

@hearty shuttle
KangaJoo and I have been testing various things with the [Freq] CC100
If we only take 3/4th of the dictionary, i.e the words up to 121824 frequency, he can import the dictionary and use it just fine, but he can't import the full one with words up to 160k frequency.
Maybe it's size issue, or there's a weird character somewhere in the last quarter of the dictionary that causes this "unknown error", or maybe something else I don't know.

hearty shuttle Jan 16, 2024, 11:14 PM

#

do you have really low ram or something?

#

the import code is apparently unoptimized

#

but it shouldn't be an issue if you have like

#

4gb at least

#

maybe even 2

#

for frequency dicts

#

but yeah i have no idea

#

best make an issue on github and the smart people can diagnose

molten gyro Jan 16, 2024, 11:17 PM

#

yeah and do report that deleting the last quarter of cc100 dictionary makes it importable for some reason

lunar badge Jan 16, 2024, 11:22 PM

#

hearty shuttle do you have really low ram or something?

I have 16 GB iirc

#

This is a gaming desktop

#

Granted I built it ages ago but ram usage hasn't really gone up that much for games

hearty shuttle Jan 16, 2024, 11:23 PM

#

wack

#

also can you open the console and screenshow whatever errors it throws while importing

lunar badge Jan 17, 2024, 1:57 AM

#

Sorry my wife wanted to hang out for a bit

#

@hearty shuttle does this help?

hearty shuttle Jan 17, 2024, 2:07 AM

#

ah just "unknown error" huh rip

#

thanks for sending the video

#

I'll make an issue later with this info

lunar badge Jan 17, 2024, 2:09 AM

#

Thank you very much

#

If you need more screenshots or something let me know

hearty shuttle Jan 17, 2024, 2:33 AM

#

oh wait

#

this is brave browser or something?

#

can you try chrome

#

ah you already did

#

damn

lunar badge Jan 17, 2024, 2:59 AM

#

yeah i actually started the process on chrome cause i heard it generally works better with yomitan and asbplayer

#

i have it set up on both right now

desert bay Jan 17, 2024, 11:23 AM

#

Why don't you try chromium

#

Marmaduke ungoogled chromium

#

Works fine

#

Basically nothing can break this browser

lunar badge Jan 17, 2024, 2:28 PM

#

Because I already had chrome

#

And didn't know about chromium's existence

#can't import dictionaries into yomitan