no

if its already downloaded I dont thnk it goes but pulling new data is limited to either 3 months, "this year" or theres a limit on what eve keeps on their end

but yea.. i think CCP refrest the data every year.. in game i only see data from the beginning of 2025..

That’ll do it. Cheers

there are history cutoffs on esi data

so you necessarily don't see all that much

hmm

and we're going up again

citadel jobs for days

sounds like you need to up your bucket again

no

it's not related to performance

it's related to how citadel jobs are handled

SeAT runs pretty well when you're not slumming it on an oversubscribed VPS

to illustrate, here I moved another VM (low load, fwiw) onto the same host as SeAT is running on...

Would it help you if we expose a control for the duration of citadel request bans+randomized the exact duration of slightly?

probably

😄

just need to wrap up a thing here, then I can put my thoughts down in a more coherent mannet

So what I see is generally: > Illuminate\Queue\MaxAttemptsExceededException: Seat\Eveapi\Jobs\Universe\Structures\Citadel has been attempted too many times. in /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/MaxAttemptsExceededException.php:24

{
"structure_id": 1041874319526,
"character_id": XXX,
"token": {
"class": "Seat\Eveapi\Models\RefreshToken",
"id": XXX,
"relations": [
"character",
"affiliation",
"token_schedule"
],
"connection": "mysql",
"collectionClass": null
},

... and it's typically stuff like this

I am not a php dev, but I spent a little time looking at the code

Let's begin from seat 4. In Seat 4, every once in a while seat ran a job that gathered unknown structures from the most important tables like assets, but far from all. Structure name loading was slow and often didn't get all structures. Seat 5 fixed that by being more aggressive. I've audited all jobs that touch structure IDs and changed them to immediately gather the structure name. This got us to an almost 100% success rate for discovering structure names, but it needs to make more requests. This is where the design of the structure endpoints comes in: It can throw an error at you and you can do nothing about it. Before seat 5, this wasn't really a problem since the load was much lower. We changes two things: First, structure jobs tolerate less errors before they stop processing due to the error limit compared to other jobs. Second, if a structure+token combination doesn't work, we put it on the citadel access block cache. As long as a structure+token pair is on the citadel access block cache, no new attempts for this structure+character are made. The thing is, no pair is on the citadel access block cache forever. I think we set it to one week. After a week, a pair is retried. Per default, the cache is in redis, so restarting redis also resets the access cache, although you can put it in the DB if you want

Using one week was pretty arbitrary. I personally don't have access to an instance of any scale to do testing, so I just had to choose something

It tries it using the the token that discovered a structure

Re this under src/Contracts/CitadelAccessCache.php it's now 4 weeks

We did consider heuristics at the time, but getting a heuristic that works for all people is pretty hard

Not having access to a system to adequately test it also has means we can't really figure out what is your current issue. While you are very open and it's really appreciated, it is not the same as having the same problem on your own machine and being able to experiment. From what I currently understand, the situation did improve and then got much worse again, right?

some changes were made to improve error handling and memory usage related to that

so I didn't have to manually wipe the cache every 24 hours or so

so what I am seeing now is a cycle where failed jobs spike on a four-ish week pattern

which corresponds nicely with the function that bans unresolveable structure_id's

That helps a lot. Since the cache timer is 4 weeks, it makes sense that it spikes every 4 weeks. What I believe is happening is that all structure+token pairs are aligned to 4 week boundaries. When the new logic got introduced, all pairs got processed at once. The ones that failed got put to sleep for 4 weeks. then they did awake all at once again, causing errors, and going to sleep for the next 4 weeks I see two changes that could improve the situation:

• Special-case the job so it succeeds even if ESI fails so that it doesn't pollute the failed jobs statistics

• randomize the delay to 4 weeks +- 2 weeks or so so that retries are spread out more evenly over instead of spiking every 4 weeks

a randomized delay will spread things out

for sure

special-case for the job might be handy

that said... hm

say you have a token:structure_id pair that has been tried several different times (and not just the ESI erroring out, but it actually giving a 403)

at what point might it be worth it to just ban that pair permanently?

reasoning: if someone leaves a player group, they will often lose docking access to the structures that group owns

consider also that we are interested in the name <-> structureid, not the token:structureid pair as such

(at least how I am thinking about it)

so the downsides are minimal imho

"ok, we tried to look up that token:structure_id pair three times across nine weeks (accounting for some randomization), fuck it, permabanned"

this would also cut down on load for CCP

I agree that in this case it would make sense. Now consider this case: A small group of players where no new players join and nobody leaves. They are in alliance A, but they have been in alliance B and still have assets/jump clones/industry jobs/anything returning structures in alliance B. Alliance B doesn't grant alliance A access. therefore, all structure jobs for alliance B structures fail, leading to permanent bans as you described. One day, they decide to switch from alliance A to alliance B. Now they have docking access, but their structure names will never resolve, even though they could.

that doesn't work

that scenario is only valid if you set up SeAT after they leave alliance B

details aside, it doesn't change that a permaban introduces its own set of issue

but in that case, a simple "clear banned structure lookups" button in the seat settings panel would solve it

imho

anyways, I've added randomization. I'll test it tomorrow and then create a PR

I could see true permanent bans being detrimental as well due to what recursive mentioned where permissions may change. Maybe the time could increase the more it fails as a way to avoid it, but if it’s spread out over 4 weeks as a randomization rather than all at once it likely wouldn’t be as much a concern anyway. As another possible thought without understanding it fully myself, is there a scenario you might anticipate where you keep an array of token IDs that have successfully returned structure ID to names in the past and prioritize checking those more frequently unless they fail, and ignore checking other tokens for the same ID if it’s been successfully updated in the last x time? For example, if 30 characters have assets in one structure, we can just check one of those and ignore/cancel the other 29 if we got structure info back in the last… 24 hours? Not sure if it works this way currently but figured I’d toss it out there as a thought. If it does just blast the request for all structures for all tokens, this might still result in high error rates for large numbers of users with assets in a structure no one has access to, but it might reduce the scenario where you have tokens for characters that left a corp or alliance no longer having access or similar.

it does check

iirc

https://github.com/eveseat/eveapi/blob/master/src/Jobs/Universe/Structures/StructureBatch.php

So think like an eve player and not logically.. People have stuff located in all kinds of places that they put there and leave or forget about, gets asset safety moved at a point. then acls change or they move groups and all of a sudden they have stuff in citadels they no longer have access to. Or you get the stranger end of the spectrum, we have a person in our group who will drop a lore item in every citadel he passes that we dont have access to.... RIP our citadel jobs.

As to the errors you see, 403 makes sense as that is ESI telling us we cant resolve that structure with that token.

The reason why the laravel log is so much more is because you have such a large backlog that you are never able to clear. So in order to allow Citadels not to consume the entire error limit available to us, they operate on a smaller error limit than the rest of the jobs. So when they exhaust that error limit there is a middleware that will delay the job. This delay keeps occuring until the max lifetime of the job is reached and you get a MaxAttemptsExceededException (which despite its name occurs on the max duration also. If you look at the failed job in Horizon you may notice that it has a large number of Attempts. This number is not the number of times we tried to reach ESI (it never tried or we wouldnt get to this point) but is the number of times the error limit middleware delayed the job from running.

One thing I am toying with is the ability for a seat admin to ignore the MaxFailedAttemptsException on a citedel job only and delete the job. The underlying behaviour will be the same it just wont pollute horizon.

The other lever we 'could' adjust is how much of the error limit we allow the citadel job to consume, If we allow it to consume more then we get higher throughput of Citdel jobs, but only to the point that we dont exhaust the error limit from other jobs

This is fair

Also seeing 13+ attempts

Hmm, TIL

Also people dropping shit all over is a reason to be able to permanently blacklist token:structure_id pairs

:v

or bonk CCP over the head...

I also kinda like jeveasset's solution where you can manually assign a name to a structure_id 😄

wish zkill still published the structure names

yeah...

that'd help

some one write a plugin to share structure names!

heh

goin back to the old days of federated killboards

I need to muster the fucks to actually look at that table of structure_id's that need resolving

I did actually think about this at some point. But I'm not sure if people are willing to share their structure names

they are not secret

in general

(putting on my nullsec alliance leader hat here for a moment) ... there's so many fucking spies and whatnot in circulation that this is not really something I worry about

They are not secret, but to my knowledge you cannot easily get a list of all structures. More like if you know it exists/where it is, you can figure out its name

yeah

that actually reminds me...is there a way to specify a specific character be used for the structure job?

like on the cli or what ever

i'll just run it with all the spies lol

that'd be handy eh?

The risk of doing it by ESI and sharing is its automated, and the name includes the system name

I have also thought about making a website like hammertime used to be (rip bookmar structure hunt) but I havent come up with a mode of operation that I can imagine leadership being happy about either

indeed

@user_131158012831203328 https://github.com/eveseat/eveapi/pull/434 Once this lands, it will immediately cause a spike, then you have 2 weeks where basically no jobs will be processed and then the randomization will start to get visible

cool

ah great, errors hit 356k and we start getting oom bullshit...

So I'm trying to chase sort of a weird issue around folk pulling tokens right before they leave corp

one unhappy server

culprit? citadel jobs

what kind of issue?

akovs security mystery

so redis literally runs out of memory

nice

(well, maxmemory is triggered)

hrm

53MB? wtf

Im guessing you have more RAM available. You could try to see by adding this line: "vm.overcommit_memory = 1" to /etc/sysctl.conf on the VM and then reboot...

fuck that

and yeah it's not out of memory

but it hit the maxmemory variable

It's always redis that oom for you right? With the build up of failed_jobs in horizon?

When setting filters. Char is director. does this auto add a ceo as well ?

Yeah

Guessing the changes to citadel jobs might help 😅

I can't tell this on top of my mind, but you could just create a test squad real quick and try it out

Depending on how bad it is, at least short-term you could switch over to the DB citadel access cache(#channel_1345031288226054245) and then starting clearing redis again

hm

let me check how that works

😄

Although there are also some other things like php artisan horizon:forget --all. They are probably the better solution

hmmmm

yeah

enabled that, also ran a general update since I saw I was behind on a few things

so let's see how it behaves

and I thought I'd already done the citadel thing... hmm

How do you uninstall a community package?

You remove it from your .env, then you restart the stack

perfect, thanks!

Sorry, I am using the bare metal install, there are no plugins listed in my .env file at all.

is it added someplace else when using bare metal?

Then you remove it from your composer.json and perform the steps you’d usually do to perform an update

perfect, thank you

that didnt seem to remove anything, the plugins are removed from my coposer.json located in /var/www/seat folder... ran thru the upgrade steps, made sure cache was cleared, even restarted the whole server... all the plugins still remain

yes I'm not an expert for bare metal. You probablly have to tell composer to actually go delete packages that are no longer used

just removing the packages makes SeAT remove a addon from SeAT, like all the config settings, files, databse, etc?

No, it won’t remove the database tables of the plugin. If you really want them gone, you have to manually delete the tables. But they don’t hurt either, so unless you are out of storage, just leave them. It is more likely you damage your install while removing them than that they do any harm

comedy option: remove them, but take a backup first 😅

I think the easiest thing is to just wipe the whole thing and start a new instance, I am just now researching and setting up SeAT for the first time to evaluate and it will just be the easier route. I was hoping it was something like AA where I just remove the addon from the config file and it disappears from the instance. But that doesn't seem to be the aspect here.

when you remove it from the compose file, it is gone from the app... but the DB tables remain

that's not really a problem per se

No, not at all, however I have removed the lines from the compose file, but the addons are still showing in the menu in SeAT and work just fine. So, I must have not been doing somethign correctly.

hmm

yeah

https://eveseat.github.io/docs/community_packages/

I may well be wrong, but you need to do some things to actually refresh the config after changing things

I think you are probably right, i searched the docs but really couldnt find anything, a fresh install was the better way to go then to mess with it

maintenance mode -> remove the addon(s) from the composer.json file, then publish ---> etc

yeah this is not spelled out

yeah, no biggies, simple fix for just evaluating a use of function idea

having a small VM for testing addons might be useful 🙂

I am running and testing on my homelab hypervisor so it wasnt too complicated

so i have some char's linked to my main account. i nee to remove them. i tried deleting them. but after a few minutes they show up again

i seem to have to delte the char several times. might be a bug or something

How did you delete them though? Did you delete the token?

I deleted them for the settings tab and from the char tab.

It might have been because they are linked to my main account

my seat instance is now stuck on this

doesnt start the web server so it returns a 404 page

im on the latest version since i took a pull in an attempt to fix it

Is there more log lines after that, as it looks like it's trying to pull in the SDE

Its resolved. My docker DNS settings were messed up so it couldnt connect to the internet =)

woops

I changed my domain on seat, updated in the .env "Seat_Domain=" And updated my callback url in the EveOnline Dev applications When I start docker, it goes through all the normal and says it starts @ my domain. But when I go to url I get the attached screenshot and a prompt from Firefox to sign in (not the Eve Online SSO) Is there anything else I need to do to swap over the domain? It was working correctly on the old one.

Are you on bare or docker

The image makes me think bare

Docker actually

I did a pull last night for the latest images

Docker doesn’t use nginx

Did you modify anything with your setup?

Negative

Then most likely, the requests don’t reach the correct server. Maybe check DNS

Well this is creepy now. I shutdown the server I am hosting this on and I still get the sign in and Nginx warning.... I wonder who's server I am hitting because that is my IP

Thank you, will go figure out what broke on the DNS side....

I'm curious how it works out for you - also planning on changing domains.

I re-did my DNS settings on the providers site but still get the same error. It is definitely pointing towards my install. I am wondering if it is something with the cert as it prompts Site is not secure then makes it go http before redirecting back to https and asking for a login. Two things of note, it had only been about 16 hours since my DNS changes, now 2 hours. The provider says it can take 72 hours. I will keep playing with it and report back.

Is there a way to trigger a new Cert to be issued?

Yes

Cert-bot

I'm unsure about cert bot and docker. But on bare metal you can reissue any time I believe

Yeah I am looking around and not finding anything like that

If you're using docker & traefik. Let's encrypt has a cooldown it will retry to acquire a new certificate after on it's own, after many fails occured. So basically you've to wait.

Gotcha, so leave the server up and running and give it some time to work itself out? Also gives plenty of time for DNS to chill out as well. Now we wait lol. Thank you

if you want to have a check on the logs

docker compose -f docker-compose.yml -f docker-compose.mariadb.yml -f docker-compose.traefik.yml logs -f --tail="50" traefik

you'll probably see the fails/retry/cooldown 🙂

or at least the reason why the certs is not valid

It is actually erroring, that was insightful. Not sure how to correct yet

All the black out matches the new domain and IP*

I ran docker compose -f docker-compose.yml -f docker-compose.mariadb.yml -f docker-compose.traefik.yml pull And that error went away, so Trafik/let's encrypt were outdated. Now it is sitting on "Testing Certificate renew" I do believe I am on the cool down. But good Steps in the right direction

New Cert was applied, same issue. Will keep playing with it

Is it still an error coming from nginx?

If so, you aren't reaching the seat stack and the request is going somewhere else

Correct, the Nginx error. Looking at the logs now. This is the only "error" I see from Seat.

That's normal. And in traefik you won't be able to get a cert because if you can't get a request to the http endpoint then you won't be issued a cert.

You need to find where that request is going and why it's going to something other than the seat stack.

Gotcha, glad that isn't incorrect ^ Will see what I can hunt down. I know it is on the host. If I shut the server down completely now, I do NOT get the Nginx error. Just trying to figure out how stopping docker last night, swapping the domain, and restarting it caused this.

Eh, if you are using traefik in production I wouldn't really say to use tls or http validation on anything you care about.. As you are publishing all subdomains you own publically at that point.

does anyone else gets these errors or is it just me?

errors started for me around 18th.

I am at a loss.... My Seat instance now works on the new domain. I just re-redid the a record and it loads instantly. So I guess it's just a time thing 🤷‍♂️ Thank you all that assisted me!

bet you have a user or role that the bot cant manage

check the connector logs (in the plugin)

yeah I already sorted this out. What happen is each time when someone leaves Discord - connector doesnt see it and tries to apply policies to a ghost member. Because that member is not in discord connector doesnt know how to handle it so that result in error log in connector and back logs. Connector as for now doesn't have any sort of blacklist for a ghost members so each time when error occurs it will just retry with next job being scheduled. After many tries it will result in errors like has been attempted too many times { "message": "Unknown Member", "code": 10007 } my solution was to just delete those users from the connector in User Mapping tab.

That is indeed the 2nd way that happens

took some time but its a fix for now at least 😄

It would be nice to have some sort of blacklist built in so if many attempts result with { "message": "Unknown Member", "code": 10007 } that char could be auto added to some sort of job excluded tab so server admin can review this and either delete or just leave it as it is with no action taken.

sometimes we want to have that info about discord username 🙂

I have completed all the bare metal installation steps, but the interface is in English. How can I set it to another language?

For just you or for everyone?

For just you set it in your profile settings. For everyone I believe you can set the APP_LOCALE variable in your .env

I manually added the variable APP∝LE after installation, but it doesn't seem to have taken effect!

I want to say that you should have it as zh-CN

Just based on our resource folders.

My/var/www/seat/lang/folder only has one en, maybe it's not in this path?

No it will be in the vendor folder

I can't remember on blade installs what exactly needs to be done to apply that change. I can imagine likely you will need to clear the cache using the command sudo -H -u www-data bash -c 'php /var/www/seat/artisan cache:clear'

Okay, thank you. I'll give it a try

DNS is a fuck

Depends what your TTL was set to.

Phew ok this a good thing to know.

Anybody have an idea why I would be still getting this error after successfully squad syncing?

You will need to share your squad-sync config for us to debug. How do you know the sync was successful? When you go to the limits in the role, do you see the characters that are members of the squad?

i was able to get it to work by manually syncing inside the terminal

As in running the sync command? Is that command in your schedule? It should be added for the squads to keep syncing

have an interesting issue why are these log files so big ? or should they be this big the last one i removed 4-5 days ago was 23 Gb

uhm

that's pretty nuts

how many failed jobs do you have?

some debug mode shit?

well i deleted the old log file and it got super aggro about it but i think it was only like 700-1k failed jobs in the last 7 days but since i deleted the 23 gb file it failed 2k in 7 days

and i cant even open it its so big

that sounds really weird

blade or docker install?

docker install

and the file is really just called "laravel.log"? Because usually it is called laravel-<date>.log

yes thats what it is allways called when i did the blade install it dated them

but anyhow when i deleted the old log file this was the most displayed error

weird

check your .env What is the log level in there?

im not exactly sure on how to check that

but if i download the file from the seat logger its only about a 1.1 gb

In your docker directory, there should be a file called .env. Check if it contains a line starting with LOG_LEVEL or similar

your right, LOG_LEVEL was set incorrectly with a typo. Set to error now. Thanks

SeAT updated, will be interesting to see how the errors spread out 😄

are there any docs on how to setup seat with a reverse proxy in my case nginx

yes 🙂

you can look at the docker stuff for examples

nginx doc

is there nothing for the dockerized version?

same link

just up a lil

on the left

how do people set up both seat and alliance auth (both dockers) on the same machine? I get the issue that both are trying to listen to the same ports (http/https)

you use a independant proxy from eithe rof them

so you have your "webproxy" compose that has a network named like webproxy-net or something

ad then you use that network as an external network in your other contianer stacks

so that webproxy has 80/443/etc open

and nothing else has anything open

hmu if you need a more indepth walk through

I believe this question happens so many times that you could do a quide how to do that in #channel_1113334797201317941 if u have a time 😂

that would honestly be appreciated, i suck at configuring all that proxy stuff xD

Hello! I'm trying to set up AFK'ing people, removing their seat squads and discord roles. I can't find a good way to stop them getting into groups or roles without me manually managing all the permission groups. Any good advice?

I think its this one of those can help you... https://github.com/eveseat-plugins/seat-squad-sync https://github.com/zenobio93/seat-discord-connector

And now I’m curious why anyone would need both. Are they not redundant in general? Or is more for specific plugins?

I believe both at this stage should work or provide access to the same data. The difference is only UI 😛 In my opinion SeAT is more user friendly though.

I currently deactivate their seat account

ill be honest, whats the benefit of squad sync over just the discord connector with normal squads?

Cant find it... Or there was never a thread..

Different use cases, I suck at UI, so I spend most of my time on AA making sure it can handle 20k users without spending money

yeah and with the CSS customization i have now done....SeAT is wayyyyy more of a plessing UX - no idea why whenever i use AA (alliance does) their nav bar options jump around..or sometimes disappear...

That means you are going between an app that hasn’t been updated

You’ll notice the whole theme changes. That’s Bootstrap3 vs Bootstrap5 TLDR update

that's ... on my list as well

welp

gotcha....i use SeAT for my corp but the alliance uses AA (I don't maintain it). But this is good to know!

So I finally got to play around with seat-discord-connector. I'm sure it has been asked before, but how can you tell it to ignore some roles?

discord bot has to be below the role that you want to ignore in roles order in your discord server

Technically it doesn't ignore them, it just errors because it can't manage said role if the bot role is below the role you don't want it touching.

Hmm, that sounds suboptimal. I guess I'll try to fix it

I put in a hacky (non-UI) workaround to get it to ignore role IDs from an environment variable array personally. I'd imagine an allowlist and/or blocklist of roles would be very much appreciated.

My idea was to get the order of the roles from the discord api and then don't try to sync the role if it is above the bot

That's definitely fair as an exclusion to prevent errors, though I think in a number of cases the hierarchy doesn't really work nicely if you use the Discord server for anything except EVE. For example, if you have community roles managed outside of the connector that you don't want showing up above named roles (for example, Corp CEO -> Corp Member-> Public/Community Role), the connector will remove those roles from anyone that is linked to the connector assuming you don't have some way of granting it also via SeAT. I'm sure for most it works just fine, but for anyone with a Discord server not 100% dedicated to EVE or with roles they'd like to manage outside SeAT, it can prove ...annoying. Previous thread on it: https://discord.com/channels/821361165791133716/821361546608508938/1181874329844187266

you cant

this method causes errors, if you have a big enough discord server with enough excluded roles you will hit up agains tthe error rate limit and get temp banned

how would i go about doing that?

absolutely not, this means your bot account cant manage the user at all

also that

I think I did something to fix that issue so skip roles I would have to check that part again.

I don't get any errors after I did that. The only errors are if someone will drop discord but it's still connected with connector

in https://eveseat.github.io/docs/installation/manual_installation/ dont work "manual installation" //note database curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | bash but if use -L all work curl -sSL https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash //note php "Next, we will have to download the new repositories GPG signing key and add it into our keychain apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 4F4EA0AAE5267A6C" before need use sudo apt install gnupg and after else apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 71DAEAAB4AD4CAB6 //note composer need apt-get install php8.3-cli for curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer && hash -r

Again a seat-connector question: Am I right in that I need to add seat-connector:apply:policies to the schedule? What schedule do you use? Also, is there any reason the plugins doesn't automatically add it when installing?

seat-connector:sync:sets will sync your discord roles with SeAT connector so you can see them in dropdown while setting up mapping users. you can always sync them manually in /seat-connector/settings seat-connector:apply:policies will sync users with their roles based on SeAT data and how you map users. you can always sync them manually in /seat-connector/settings as well. > Also, is there any reason the plugins doesn't automatically add it when installing? well, good question 😄

I run my a bit hotter

with ts and discord hooked up

ill be honest I set it up so long ago I dont remember it not being automatic

tbh I dont think if u need to run seat-connector:sync:sets that often... if you really need it you can do it manually but otherwise its just there to refresh from time to time 😄

hello, The seat is installed via docker on the old server. I want to move it from old server to a new one. Is there any instruction on the transferring?

Not really , but all you need is the .env file from the old server and a recent backup. Set up seat (Docker) on the new server > replace the .example.env with your .env from the old server > start the stack Then you can extract the database into the new setup. And change DNS records after if the new server has another ip.

https://eveseat.github.io/docs/installation/docker_installation/

https://eveseat.github.io/docs/admin_guides/docker_admin/#database-backups-and-restore

where to change DNS records?

That depends on where you bought your domain

I setup with a domain

Also, contributions to the documentation are very welcome. This question is asked often enough that it probably warrants its own page

oh, yes, I have parsed my domain to the new IP.

stupid question the seat notification A member has not logged in for some time! Check corp tracking. what is the alert called in the Gui?

"A member has been kidnapped send help"

heyho, i have basically 2 problems at the moment. One is a toon that joined corp yesterday. I was able to do my vetting as usual, but since he joined only his public scopes update so any roles / squads / discord roles i set for example from corp titles won't update, same as mining ledger etc etc. We already tried deleting and re-registering the toon, revoking access for seat in his account management and creating a new thing and so on. One thing i noticed is, when he re-registered after deleting everything he wasn't even assigned to the default / public squad that auto adds everyone to assign minimal roles. But yeah, so now SeAT recognises him as in the corp, but isn't updating anything else. Did i mess something up by deleting and re-registering or is that a token issue or... i don't know 😄 it didn't update for about 16-17 hours now. Edit: When viewing his account under scopes, it shows the full thing The other issue is performance (and migh be related?). When the cronjobs are running, the website gets very slow and unresponsive sometimes. But when i access it during the same time from a different pc it works like a charm. All that while it's using almost none of the allocated ressources. It's running in docker. thx for your work guys ❤️

So if you say the scopes are okay, the next question is how do his jobs look like?

• are there any jobs related to his character being scheduled?

• do they succeed?

Regarding the performance issues, your description of performance issue is a bit too vague to do anything with that information. What makes you think it is slow?

the browser keeps showing the loading animation for minutes upon minutes. and even though the bottom info bar says the site was rendered in 0.x seconds its not completely displayed.

trying to find out about the jobs right now

that might sound like a dumb question and i thought i was smarter than that xD is there an easier way to find out about his jobs than scimming through completed jobs in horizon? It does show completed jobs for his character id though

right, i've put the ID on monitoring and now jobs are popping in there. Including title but the titles didn't update

not showing any failed jobs

what makes it so strange to me is, that we had another recruit shortly after this one and for the newer one everythin is working perfectly fine

on the alliance's auth he updates like normal too, just verified that.

i managed to somehwat tone down the "performance" issue but toning down the frequency of bucket updates, however i am ending up wiht outdated tokens so will have to experiment a bit until i find a good balance. still nothing new on the dude who's data isn't updating. I poked our alliances auth admin about it if he saw something similar and all he came up with was that this happens when people pull their scopes but i assume checking their scopes in SeAT would reflect that?

yea

so I had a single instance of one dude that seat could never add

when I dug into it, the character object that the eis returned wasn't right

like, just malformed a bit

opened a ticket about it and it got fixed with no reponse in the ticket

so it might be on CCP's end then? i started to suspect that.

what kind of bucket update schedule are you on?

i was at every 10 minutes which boosted performance significantly but tokes weren't keeping up so i am trying every 5 minutes now but sett it off to 2,7,12,17 and so on

originally was at every 2 minutes

ohhhh

yeah that might get fucky with how cronjobs work

how many tokens and what kind of server are we talking about here?

you mean the offset might make it worse? my idea was to move it away a bit from other jobs

it's a docker running (i'm a bit ashamed of admiting this :'D) windows 10 for the time being behind a cloudflared tunnel

heh, how many tokens and what sort of disk

60 characters atm, m2 disk. the machine really isn't running anything else other than a backup software, docker and remote desktop

are your jobs backing up?

like large queue

not at all

can you post a screen shot of your horizon page

/horizon/dashboard

i'm off to FCing for now though^^ will check back later

yeah... don't do anything fancy with the bucket update schedule, set it to */20 * * * * or something and call it good.

yea your box is totally chillen

is this where I can ask general questions about setting up seat, or would that best be put in support-forum?

yeah t always was but still, sometimes when a lot of jobs are running sites don't load so well. They render super fast but the browser keeps showing loading animations and you can't click stuff

Hey all, brand new to running SeAT and Auths in general. I've gotten everything up and running it appears and the last hurdle I have is learning how to manage permission. How do I give a global permission so everyone can see the "Connector" sidebar so they can link their discord accounts?

create a new role, call it say....Auto-Roles, give it the Permission Seat-Connector -> View Create a new squad call it....Auto-Roles, add a filter for your corp/alliance and have it assign the Auto-Roles role

awesome thank you, because we have a bunch of out of alliance/corp individuals coming and going from the discord as well can I do a general one for everyone that joins?

sorry that was copy pasta

if you dont care who can join your discord

and I mean at all

I'm in Faction Warfare so we have new people accessing the discord all the time

just edit the filters on the auto-role squad to match your needs

you can add the filter faction is angel cartel

or what ever

just make sure seatbot is above all your other roles

and use seat to manage your dicsord roles

Any chance you are using docker desktop? Or is it docker in wsl?

docker desktop

That will likely be your performance problem.

We recommend the use of docker through wsl as it is night and day there performance difference

oh really. i'm completely new to the docker world. Used to host VServers in the past btu thats... long ago 😄 allright cool, will look into that then, thx for the call ❤️

Got everything up and working, thank you for the help

docker desktop variants run a vm to then run the containers nested.. vs docker engine on linux is native using namespaces. So not virtualization layer.

And on say windows you are adding additional layers for the windows filesystem and translation for the network drivers and of course pre-allocation of ram

Aye, makes sense. I didn't expect it to cause this specific issue but after the hint it might be from there I found descriptions of the exact same behaviour I am experiencing on other projects so that's gonna be it. For will have to live with it, maybe when I'm on vacation I'll take the time to swap OS

Allright, about the character that I can't update info an. He added 2 more characters and the same is happening but it's working in alliance auth. So I doubt he's doing something weird to prevent intelligence. One of my recruiters said they tried to delete and re-tegister him on seat to force an update on the token so he could get discord access. Could that somehow mess up his database entry and prevent his data from updating? Since it seems to be seat user related, not character.

Player Trading doesnt track what has been traded right?

right

afaik

you can only see it happening

In theory nothing done on the web ui should be able to stop the characters updating. So couple of questions. If you monitor the characterids for those toons in horizon do you see failed jobs? Second thing is are you using any of the rules based scheduling? What does you schedule look like? To try a manual fix as a one off you can try running `docker compose exec front esi:update:characters {characterid}` swapping out character id as required

thank you for your message. it isn't showing any failed jobs for any of his toons, but i notices all jobs show a 0.00s runtime, while jobs on other toons show at least some time. I am not using any character based scheduling. running the command returns

OCI runtime exec failed: exec failed: unable to start container process: exec: "esi:update:characters": executable file not found in $PATH: unknown

same return with different character IDs of toons that work. I didn't expect it to work with a different ID with that error message but tried to be sure.

is this with it running on WSL?

or docker for windows?

docker dekstop on windows

🤢

yeah i understood that, still seems weird to me, that it's workin on all other toons, only not working on toons related to this specific user on seat

even a linux VM would be better than ... that.

well it clearly cannot do the thing, so

0.0s runtime matches with not updating. Usually, a ESI request takes at least 0.1s. But I can’t say what is wrong

and that error message indicates an environment issue

No the command just misses php in the command. It should be … exec front php artisan seat:…

aye, noted. and i already noted earleir i'll probably migrate to another OS when i'm on vacation. i still have doubts when every other toon works it's related to that. BUT i'll fully admit i might just not realise 😉

If most work I agree that it is most likely no OS issue

stuff like this makes me happy I'm not a programmer 😅

haha, that's the weirdest thing for me right now. I used to be on my way to become a programmer but then took a completely different career. Now that i'm dealing with this - and this might sound strange - i kinda have fun with toying around with it and trying to find the soultion even though i feel like i'm dumb for not getting there yet 😂

says php service isn't running.

Yeah I’m typing on a phone, it is not the complete command

ah ok, yeah let me look into that myself 😄

fully understand, i dropped my phone and swapped to seamdeck with wireless keyboard for this while i have some downtime right now. i might vanish if things start happening around me 😂

docker exec -it seat-docker-front-1 bash -c "php artisan esi:update:characters <replace_with_id>" try this

if u get any error paste it in here

that did it with some editing of the container name. says processing character with the correct character name but web interface still shows no data

give it some time

yeah you will need to give it some time but in a meantime go to horizon + logs and check if u get any errors while this being processed

no failed jobs so far. the ones showing up at least show 0.01s runtime now 😄

allright, downtime over, gotta go back to work for a bit. maybe it's updated then but currently i have my doubts tbh. Shows jobs running through but no new data when viewing the character in the web interface. as if it's looking at the wrong place or something... thanks for trying to help me guys but yeah, gotta to back to work for now

Does anyone have any ideas why this guy auth to CCP isnt working? https://discord.com/channels/821361165791133716/1371744681855029268

ok so.. jobs seems to have finished, isn't showing any failed jobs but the data shown when viewing the character is still not updated - except for public data, yet when checking which scopes he has the userpage tells me he has full scopes. I don't know, by now i think whe i'm off work and see him online i'll delete his characters and ask him to re-add them step by step with me, see if he did anything wrong maybe? it's so weird to me 😂

Would you mind setting the logging level to debug, rerun that command, wait until the jobs "succeed", turn debug logs back off and share the logs?

In your .env, there should be a LOGLEVEL key. You have to set it to `LOGLEVEL=debug` for debug logging

Sure I can do that, might need a bit though, nightshift and it's getting late, this is where the fun begins 😅

hope i did that right. In the laravel one i already notice something about scopes not matching but as you can see i the screenshot it says he has them all - unless i completely misinterprete that.

hello, anyone know which schedule can update table bucket_refresh_token?

what is your problem?

I found that some new characters are not updated in a timely manner and they are not in the bucket.

Would you mind sharing the character id of the character in question so I can search the logs? you can also DM it if you don't feel comfortable sharing it here

as in no jobs are scheduled for affected characters at all?

looks like need wait some days( after 4-5days) can have job for these new character.

that sounds very wrong

That sounds more like a schedule problem or overloaded server. What is your schedule?

can I pm to you?

Sure, if you don’t feel comfortable sharing some information here you can. But then also others won’t be able to help

The schedule is not sensitive at all though

Ok, so it isn’t the schedule

So does it happen for all new characters or are we talking about a single character?

i just suddenly started getting hundreds of this error

anyone else?

sure, should have thought about that 😂 2114049750

same...

a few at least

Noticed a few as well with this exact error a day or so ago, dug around in the tables a bit and found the corp ID the character was part of was not existing in corporationinfos per the error (so showing non-clickable corp name in the character sheet), ran a manual `php artisan esi:update:corporation {characterid}` for each referenced character which pulled the info for the corp, roles updated just fine from there and all good from then on.

eww

that doesn't sound like fun

I only had two character IDs experiencing the issue so it looked worse than it was.

The interesting thing is why it only started suddenly. Were they new auths? Also if people could DM me with character ids and Corp ids that this occurs for this would be great

Hi, struggling to setup my SeAt, I have a docker install with Taefik, keep getting blank 500 internal server error when trying to access after booting, also read that I'm meant to provide Laravel log to assist

Looks like you haven't set your app key in the env file

You can set one with the following command sed -i -- 's/APP_KEY=insecure/APP_KEY='$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c32 ; echo '')'/g' .env

Ah read down through the install guide, must have ran over than bit, assuming I just run

 $appkey = (-join ((65..90) + (97..122) | Get-Random -Count 32 | % {[char]$_})); (Get-Content .env -Raw) -replace "APP_KEY=insecure", "APP_KEY=$appkey" | Set-Content .env

What's the difference between these 2 commands?

One executed in Wsl and the other windows?

Yeah, Linux (wsl) vs windows

OK, worked perfectly, thank you, just SSOd in with my own character and notice the total character isk is 0, assuming this is because it is running through all the character info bit by bit?

Yeah will take a bit to get everything. You should be able to monitor it on the horizon dashboard to see jobs turning over

Yea managed to find that, this thing is amazing haha, I'm a C

dev myself and it would take me forever to build something like this but then you remember how long eve has been around and realise lol

Ok everything is working as expected apart from the market browser, nothing is appearing in it

i am geting more characters that aren't updating even though it doesn't show failed jobs and they all show full scopes. One right now doesn't even update public info like corp

Does anyone else have so many timeouts errors? since 18th I get like 200-300mb logs mainly because of that ... Did esi shit itself again?

Thanks for the reminder, I forgot about it. So in your logs, I see a lot of jobs being removed due to this error

[2025-05-21 21:04:36] local.WARNING: [Jobs][Middlewares][d8388212-9c70-46ad-a737-240ec608f985] Check Token Scope -> Removing job due to required scopes not matching with token related scopes profile. {"fqcn":"Seat\\Eveapi\\Jobs\\Industry\\Character\\Jobs","character_id":2114049750,"scopes_profile":1} 

A quick question: are you on the latest seat version? There have been a few SSO changes in the last few months, I just want to amke sure I'm debugging an actual problem

Also: Did you change anything related to scope profiles recently, especially deleting a profile?

It seems that our SeAT instance does not automatically update alliance contacts. The alliance contacts table in the DB was entirely empty until I manually refreshed alliance contacts for a specific alliance by clicking the refresh button in the UI. From the above, I deduce that the mechanism for pulling the contacts (and the relevant tokens and scopes) work as intended, but that the job is somehow not running on a schedule. We have the default esi:update:alliances 25 6 * * * set. The metrics for Seat\Eveapi\Jobs\Contacts\Alliance\Contacts show just one job (the one I triggered manually). Does anybody know what is wrong / how to fix it?

no recent changes to the SSO stuff. i do have 2 profiles though, one for Member, one for Public. They use different log in buttons on a custom sign in page, i'm not sure how to identify whihc one is the current default tbh

and don't worry about forgetting it, to my understanding this is a free time / hobby project and not anyone's main job aroun dhere. i had a crazy week at owrk myself so didn'T even spend much time with this after sending the id

So i turned on the wardec but unfortunately. the first time i get a notification, it say "unknown" for the aggressor... it was pushed from an alliance level, not sure if that is the reason why...

Would you mind connecting to your database, run the following two queries and share the results:

sql
select scopes_profile, scopes from refresh_tokens where character_id=2114049750;
select value from global_settings where name="sso_scopes";

will take me a bit, for some reason connecting to the database didn't work. I'll have to look into that but RL busy for a while first. Will get back to you when i can

How do the SSO profiles work for the scopes? I can see I can make multiple profiles and configure the scopes but what is the actual application of this?

i have one that only asks for public scopes and one that asks for full scopes. Public is for people not in the corp we still want to give some "special" access to specific channels in our discord or dudes from our alliance but not corp who get to see our market channel or a special friendls chat etc but not the actual corp only channels. Full scope for members

How do you switch from one to the other? where is that configured as when I try sign in as a new character I only ever get the same one which asks for all scopes

On the page where you create the profiles on the right you can make a custom login site. I made one with 2 buttons, one for public one for members.

Ohhhhh, I see it now, amazing, thank you 😄

Make one that is like public access and people can join the discord like allies and such and be tied to squads with discord connector for example.. if you ask an ally that's not in your group to do full scope auth that's unlikely going happen

I'm trying to do the custom login page styling process but it doesn't seem to be working at all, I followed this whole process through https://eveseat.github.io/docs/styling/ (I have a docker installation)

https://discord.com/channels/821361165791133716/1247605206372323359/1247605206372323359

try this

The thing that confuses me about this is that I don't have an opt directory as it keeps going on about?

OK then adjust to yours

What directory do you have?

this is where I have been modifying things such as .env

And also attempting to do this styling change

um... is this windows docker ?

yes

I cant help with that I'm afraid

but what part is not working for you?

it seems like you have the same structure just different dir

so u have to adjust I would say

Yes the guides are written for linux. You'll have to notice what is different for windows and adjust yourself. You can usually just replace /opt/seat-docker with the directory your docker-compose files are in

Would I need to fully qualify the directly for example starting from C: or do I go from currently directory like /seat-docker-master/custom ?

I still get the unknow for the character name in the corp application notification sent to discord, any news on the fix?

just use linux path

like ./ and youll be fine

Hey all so I have a tricky problem and I cannot figure out what to do. -- A corp member with 3 toons left to join another corp. -- He then came back to SeAT for tax reasons and added a new "user" instead of linking to his main. The toon was already linked to the main (but missing tokens) so he were able to add this toon as a new "user". I deleted that new "user". -- Currently the tokens are all valid but he has left with all his toons to another corp. Yet, I am getting failed jobs upwards of 1500 in the last 3 days just for this one single toon on everything from notifications to contracts to etc. etc. I tried deleting the character and also deactivated his main toon but none of that is working. Should I delete the "user" and then go delete all his "characters" as well? I was keeping him around for tax compuations at the end of the month.

failed jobs for what reason?

this is in the last 2 mins

meanwhile all his scopes seem to be valid for all 3 toons

Maybe he removed scopes?? so my only option is to completely delete the user and all characters i suppose?

this machine is not happy about being told to shut down 😂

well... 12 more seconds and proxmox yanks the power from it

there we go

🤔

May 29 19:41:57 tnt-seat-2024 systemd[1]: supervisor.service: Consumed 3w 3d 14h 52min 2.336s CPU time.
May 29 19:41:57 tnt-seat-2024 systemd[1]: Stopped supervisor.service - Supervisor process control system for UNIX.
May 29 19:41:57 tnt-seat-2024 systemd[1]: supervisor.service: Unit process 2594342 (php8.2) remains running after unit stopped.
May 29 19:41:57 tnt-seat-2024 systemd[1]: supervisor.service: Unit process 2594341 (php8.2) remains running after unit stopped.
May 29 19:41:57 tnt-seat-2024 systemd[1]: supervisor.service: Unit process 2594338 (php) remains running after unit stopped.
May 29 19:41:57 tnt-seat-2024 systemd[1]: supervisor.service: Failed with result 'timeout'.
May 29 19:41:57 tnt-seat-2024 systemd[1]: supervisor.service: Main process exited, code=killed, status=9/KILL
May 29 19:41:56 tnt-seat-2024 systemd[1]: supervisor.service: Killing process 505 (supervisord) with signal SIGKILL.
May 29 19:41:56 tnt-seat-2024 systemd[1]: supervisor.service: State 'stop-sigterm' timed out. Killing.

did this already and worked fine all apart from the logo :/

and couldn't be bothered to fight with it any longer so I've left it since

What are you using for logo? Link or downloaded image?

Imo don't host things for long term on windows

I removed (deleted) the character not the user (which is a different toon) but then the character came back?? Why Is this possible?

I tried both downloaded image and url and neither seemed to work

Is it possible to make users accept a required scope such as the public info but then optionally enable other scopes instead of making them all required?

You can have multiple profiles, but no, there is currently no way to let a user choose

you can have multiple profiles? like one for corp and one for guess?

does that need to be my public IP?

or my local ipv4

Local

so change it to http://100.85.199.85:9443/