Error Failed to build query A : [sqlstore.max-retries-reached] retry 1 : database is locked

[graceful lance]

Hi Team

I’m using NAbox on that I have grafana I’m running this mentioned subject as error failed, any help

Thank you

[trail river]

Hi @graceful lance , Please upload a support bundle to https://upload.nabox.org/kuhy-maqo-fiba and we'll take a look

[graceful lance]

Thank you but in NAbox I do not see button to download support logs

My NAbox version is 3

[trail river]

thanks. can you ssh into nabox and run dc logs grafana 2>&1 | tee grafana.log and upload the grafana.log to the link above?

[graceful lance]

@trail river thank you I have uploaded log as nabox_logs_error

[trail river]

thanks for the log file. There are lots of errors about undefined variable %!q(MISSING) when parsing your alert rules. Can you upload your alert rules to the same URL as above?

When did this problem start happening? Reading the Grafana issues page, restarting Grafana sometimes fixes the lock issue you are seeing. Have you tried restarting nabox via dc restart?

[graceful lance]

I have updated this alert it was my mistake that I missed s in labels.instance -

Do I need to run only dc restart

[trail river]

yes, please try that

[graceful lance]

Hey thank you as I will try at 04:30PM ET and will update you

[graceful lance]

I started that but still it’s starting

How can I solve this

[graceful lance]

@trail river cadvisor is starting and going on

[graceful lance]

Even if it’s restarted successfully now but still I’m seeing the same error database is locked

[trail river]

@graceful lance did everything successfully restart? Can you upload the grafana log file for today by running dc logs grafana --since 2025-01-22?

@tacit dust have you see this before?

[graceful lance]

I will upload the logs on same location

[graceful lance]

@trail river I have uploaded the log

[trail river]

Thanks for the log files @graceful lance. I agree that there are still intermittent database is locked errors with retries. Looking at Grafana's issue tracker, many folks have run into this issue over the years. Out of the box, Grafana uses SQLite to manage its configuration, alerting, etc.

It looks like Grafana isn't doing everything they can to use SQLite in the most efficient manner and there are some upcoming changes to improve things https://github.com/grafana/grafana/pull/98359/files
https://github.com/grafana/grafana/issues/16638#issuecomment-2101570173

You can try applying some of these config changes yourself if you want.

It also seems like some folks give up on SQLite and migrate to MySQL or PostgreSQL https://github.com/grafana/grafana/issues/47296#issuecomment-1192739610

[graceful lance]

Thank you so much for help , but I’m wondering If we migrate to NABox version 4 , should it be resolved

[trail river]

your welcome! There are other good reasons to migrate to 4 so I agree that would be a good place to start. If the issue persists after migration, we can help you try some of the ideas above if you want

[graceful lance]

In the meantime , can you please help me to know where I can enter GF_DATABASE_WAL= True as I’m trying to find grafana.ini on NAbox but no luck

[trail river]

@tacit dust

I don't have an nabox3 instance around to check, but can you sshing into nabox and run this? It may return nothing, in which case, the grafana.ini is not being volume mounted and we can find it inside the container
find / -name 'grafana.ini' 2>/dev/null

[trail river]

Here's an example with Nabox4, I suspect the same will work for Nabox3

[graceful lance]

I believe it’s NAbox is container base so all will be inside containers

[graceful lance]

I checked

And went inside container

And run ps

And I found the ini file

But whatever I did change I do not find like SMTP host and all

But when I run inspect docker I see entries as sent in dm image

[tacit dust]

Try and use environment variables when you can

to define GF_DATABASE_WAL you will have to use a docker-compose.override.yml in NAbox 3, following this method https://3.nabox.org/faq/#configure-email-server

[graceful lance]

Thanks Yann I appeneded the GF_DATABASE_WAL into the docker-compose.override.yml and restarted the dc up -d grafana

[graceful lance]

And will update you in next two days

[graceful lance]

Thank you after appending this I’m not getting any error of DB locked thanks a lot