SeAT - 💬-general - Page 4

I think notifications and journals contribute huge part into this

I was able to shrink 107k tokens turnover time from 80hrs to 25hrs

now my server can queue 105514 tokens in past 24 hrs

Originally I did this because I had lots of duplicated primary errors

realized firstOrCreate is actually not atomic

Interesting.... I will have to have a look into that at some point

As in is there a laravel function for those inserts? Or is that something you wrote raw?

I use the already deleted CanUpsertReplaceIgnore trait that was once in old version of eveapi

it is MySQL syntax though… things are different in case there is plan to move to PG

oh and a good thing on that trait - I can do batch import without waiting for n+1 responses

that way I chunk data to 100 and insert all at once

what kind of storage setup is this? ZFS or something else?

ZFS on 14 x Seagate Exos X22

raidz-something?

ah

Zfs doesn't like to have a large pool like that. You might want to consider spliting into two pools instead.. 7 disk pool..

hmm thats a new thing to know

eh, that depends

but running databases on spinning rust is a bad idea

what kind of nvme drives are those, and how big are they?

8TB KIOXIA each

https://www.youtube.com/watch?v=h4ocFY-BJAQ

What this video

honestly it works (write and well indexed read) until it doesn’t (full table scan)

yeah

so those kioxia drives...

how big are the partitions for log and cache?

36G for log, rest for cache

you have 7TB of cache...?

but those cache doesnt really work tho

yeah no shit 😂

how big is the SeAT VM?

the reason why it does not work remain unknown

l2 header size stuck at 0

there is like 800 GB db

right

less than 100G code I think

I'd nuke the cache

the main reason I dont put it on SSD straight

I once had a SSD install and it wears at light speed

I might did smth wrong at some point

ehhh

maybe

what's the kioxia drive model you've got?

ok.. to understand right.. you have the Seat VM on the 14 zfs pool?

KCD8XRUG7T68

that's what it sounds like

yeah

theres like 40 more other VM on it as well

okay so

ok.. having that large pool will have performace issue.. The video above explains that why.. and the best use for it..

remove the existing cache partitions, make new ones, say 200GB on each drive, then make a 3TB partition on each drive and make a raidz mirror on those for SeAT

your drives are rated for 3 DWPD and with ~50% overprovisioning they will handle that just fine

(plus you're already doing slog so the SSD write load is not going to change much)

i actually only notice very minor write load on that

ah seat vm has cache: metadata on it

ah yep

then your l2arc isn't going to be happy

🙂

the l2arc has problem in its entirety i think

no, you've set it up wrong 😄

i once remove it from the pool and readd them

and now header size keep at 0

the only difference between that would be mfuonly=2

yeah

I'd do this

and see how it goes

are you referring to the catch pool?

i wont pick this machine type if i were to do that😐

wut

I'm referring to the l2arc he added

i buy this 14 hdd metal for the sole reason of putting seat on HDD

??? the L2ARC is the ram... how would you remove that?

no it's not.

ARC is ram, L2arc is on disk

so?

why i would move it back to ssd

if you wanted any sort of performance on spinning rust, you'd be using mirrors, not raidz

I dunno... to get performance?

what kind of SSD did you use before you moved to this machine?

samsung MZQLW960HMJP

i have no performance issue in 80% use cases on this hdd setup

it just makes bad db decision super apparent

heh..

I think we know what the problem is 😛

also that's a very small read-intensive drive, as far as I know, no wonder it didn't like the workload...

that might explain why the hdd actually makes it work faster than that ssd setup

how much memory do you have?

256G on this

apropos arc vs l2arc

64G allocated for mariadb

64G for ARC

ah

I'm having a problem with the SRP module. We implemented the Janice API, which I had, but the SRP value was much higher than the value I manually checked on Janice. I won't even mention the fact that with 505 registered characters, the seat uses 24 GB of memory ^^

how

how the f

this is with several thousand tokens...

To the SRP module. Can you share a killmail and your rule setups that you have observed this for? Also 24GB of memory seems way too high, I feel like there is something else at play there

you gotta pinpoint who is using memory

my 170k registered character instance uses less than 10G

https://zkillboard.com/kill/131729862/ I don't know if this is an API issue or something with a misconfiguration.

Can you share what your SRP is configured for

There is also the test page that can be used to show the stats we need too

i sended in your DMs

I'll have a look in a bit.. Normally dont check my DMs due to qty... At least you will be at the top for now

I noticed because I tried to contact you a few months ago also through some friends ^^

Yeah my DMs are generally constantly overful so get mostly ignored

am i like.. blind or is the optio to impersonbae a user to check permissions gone? 😄

It is not gone.. So long as you are admin you should have that ability

i have question becuse i can upgrade plugins below version 5.0 and dont know why?

Plugins might not follow the same numbering scheme as the seat core

okay thank you i repair

i have another question becuse after updtade crypta say to me to delete that old kilmail but it not working

Try first issuing

sql
DELETE FROM killmail_details WHERE killmail_id = 131729862;

then

sql
DELETE FROM killmails WHERE killmail_id = 131729862;

Sorry extra delete froms.. Bad copy. updated

Done 🙂

i test srp and looks like that

Check your browser console for any error logs. But that usually means that you dont have some data in the DB up to date

okay i write to priv to not spam

Lets just head to #channel_821361546608508938 as the best place for the convo