SeAT - forum: Ajax error, when searching through large database - Page 1

Nothing in nginx, php, redis, seat logs appear to show any signs of the reason why the ajax error appears. A web browser's debug console returns POST : 504 (Gateway Timeout) as the only hint.

I have also attempted to increase the timeout for php in php.ini with

max_execution_time = 300
memory_limit = 512M

for the .env file, i tried

ASSET_QUERY_TIMEOUT=300

and for /etc/nginx/nginx.conf I added

proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;

I‘d be interested to know how long the browser tries to load before the 504 happens

A screenshot of the error in the browser would also help. It usually saves some back and forth as it gives more context.

errr yes seems like a html return

thats fine

I can read html

i'm going to change that file again but yeah

so you have cloudflare in front of your seat, right?

yeah

you think its the problem?

so increasing your php and nginx limits didn't help because you are running into a cloudflare limit

the default for seat is a 30s timeout. You increased it to 300s, but we see 60s from cloudflare

ah shit ok let me try

didnt think cloudflare throttled

anyways, to get to the root of the issue: I just wanted to see what was causing the timeout. is it a proxy saying long enough php, I'll just respond with 504 myself or is it laravel throwing the error

just one last thing: do you have a character with fewer assets to test it, for example an alpha alt you never really used? Just to make sure it is really related to the amount of assets and not some other wierd thing

so the weird thing is i ran the search again and it sometimes goes through and it sometimes doesn't but laravel doesnt show any errors related to the asset search

it does not error with a toon i don't use often no

as for the first, i searched apostle on 3 toons, it ajaxs, i changed the search from apostle -> sleip it errors, i add a new toon onto the list of characters to search while the filter is under sleip and it doesnt error

your last few messages combined really make it sound like a database query for the search taking too long. This probably can be optimized, but it is probably a bit more complicated + I have no clue how anything but the most basic datatables stuff works.

i read previously you can attempt to "reindex" your database?

it is nothing you can easily do as a user. Index are things you can add to tables (like e.g. columns) that make some queries faster by having optimized datastrucutres for this particular query

would having too many laravel process queues also throttle the search too much?

what makes is even more complicated is that the assets themselves don't store the name of an item, only it's type id. this means just to get the name it already needs a join

They can be related, but this is not the solution. If your CPU load is high, less work can be spent on the search before the timeout hits. However, when you remove queue processing power, CPU load might drop, but your queue will just fill up. Having too many workers is also not harming performance as an idle worker doesn't consume noticable resources

i'll try lowering the queue a little bit to see if that will "help" but i guess the database just really isn't optimized? that or cloudflare is timing us out?

yes, it boils down to the database not being optimized for search. I'm not sure what cloudflare is doing. it could time us out, or it could just replace the error page

ok ty is it possible to put this here on hold for now, and @ you for updates if there is new info?

can you confirm if this is true?

Cloudflare doesn't trigger 504 errors on timeout
504 is a timeout triggered by your server - nothing to do with Cloudflare.
524 is a timeout triggered by Cloudflare.

I don't use cloudflare

You could try bypassing or pausing cloudflare for testing and see if the timeout behavior changes, at least then you'd isolate a bit further - but cloudflare docs are here on that: https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-5xx-errors/#error-502-bad-gateway-or-error-504-gateway-timeout There could be a scenario where cloudflare is the problem with a 504 but you'd need to check response data (in the response tab as the message I replied to) to see it, I'd imagine, since otherwise the ajax error will be all you see. Their timeout for cloudflare is 100 seconds by default, but you'd normally get a 522 or 524 if that happens.