tooldev-general

[simple ravine]

hehe

[simple ravine]

aha...

src becomes manipulated by the Ooz_Decompress

I thought I was becoming mad.

but only the first time around

strange

[worthy cape]

Eh?

[simple ravine]

    internal unsafe class LibOoz
    {
        [DllImport("Bundle\\lib\\libooz.dll", CallingConvention = CallingConvention.Cdecl)]
        public static extern int Ooz_Decompress(void* compressedContent, int compressedLength, void* decompressedContent, int decompressedSize,
            int fuzz, int crc, int verbose, byte[]? dst_base, int e, IntPtr cb, IntPtr cb_ctx, IntPtr scratch, int scratch_size, int threadPhase);

        public static int Ooz_Decompress(Span<byte> compressedContent, int compressedLength, Span<byte> decompressedContent, int decompressedSize)
        {
            void* src = Unsafe.AsPointer(ref compressedContent[0]);
            void* dst = Unsafe.AsPointer(ref decompressedContent[0]);
            return Ooz_Decompress(src, compressedLength, dst, decompressedSize, 0, 0, 0, null, 0, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero, 0, 0);
        }
    }

sorry for the long lines

but the data in src becomes mutated for some reason

I'll check in a bit what changed, running out of nicotine here, so need to go to store

[worthy cape]

@simple ravine Verified, decompressed with actual read-only pages and it went bada-boom.

Backtrace: https://gist.github.com/zao/674a507ce929522be3d81017c038d72b

I'm on Linux so I can't tell if the closed-source decompressor does it as well.

I guess that there needs to be documentation on that you either need to copy-on-write your file mapping (MAP_PRIVATE) or decompress from a mutable buffer.

[simple ravine]

fuck, that means I need to copy the content

[worthy cape]

I'm curious to whether the block is safe to re-decompress after the first go.

[simple ravine]

it is

partly

at least

first I just checked the first 20 and last 20 bytes to see if they were the same, and they were

so i started to believe my code was shit somewhere

My thing is a 2-step decompression process, first it decompresses the _index and then goes to check on the inner stuff

[worthy cape]

Time to heat up some dinner and see if I can survey this with all blocks.

[simple ravine]

see how the 2nd and 3rd time the sha256 is the same?

much strange.

[worthy cape]

Did you inspect the output tho?

[simple ravine]

No, that's what's crashing, the BundleIndex.D(dst) will read the stuff

haven't gotten to the store yet, showered and about to go there, will be back in ~20-30

[worthy cape]

Enjoy the weather 🙂

[worthy cape]

Hah, the game doesn't want me to play it today:

85425296 31cb [CRIT Client 7080] [VULKAN] vk::DeviceLostError::DeviceLostError: vk::Queue::submit

[simple ravine]

Did the drivers to a hickup?

[worthy cape]

Been having the game just freeze and drop to desktop, like once a day or so.

Seems to be NV drivers faulting or lost Vulkan devices, so yay, could be anything.

[simple ravine]

ah, so probably what happens is that the drivers eject itself when it freezes

does other things also seems to "reset" like desktop manager / other windows apps?

[worthy cape]

Nope, just the game cleanly dropping out.

Heard a loud "clack" from near my computer when it first crashed today, so heaven knows what's up 😄

Guess it's haunted.

[simple ravine]

ouch.

[worthy cape]

Running on drivers from late July, so probably nothing to lose by upgrading... could code in the meantime 🙂

[simple ravine]

omg why on earth would the damn thing not be idempotent

[worthy cape]

Makes sense to use the input area for scratch space if you don't want to separately allocate some.

For particular values of "sense"

[simple ravine]

ahh wait, there is an IntPtr for scratch here

wonder if I allocate something there, it won't touch my damn source

[worthy cape]

Note that ooz doesn't have the argument, only actual oodle.

[simple ravine]

yeah, it did nothing

[worthy cape]

The wrapper export I made discards all the tail arguments, only there for signature equivalence.

[simple ravine]

ok

now I wish I had a oodle to compare with

[simple ravine]

@worthy cape your 0.158 does that include FNV1a hashing?

[worthy cape]

Nothing to hash, it's just for decompressing and reading the information in the index bundle and directory bundle.

[simple ravine]

I think I need a better algorithm for mapping reading the filenames

My 200ms read the outer and the inner bundle and grabs the paths etc

[worthy cape]

0.85s includes building acceleration structures from hash to directory and hash to file entries.

Also generating and printing all possible paths puts it at 1.96s

[simple ravine]

https://gist.github.com/andreandersen/ababc38a224d67680855d01cad1ee3b0

this is how I do it, do you have any pointers or suggestions on how it can be done smarter?

the constructor of PathString will FNV1a hash and store the name and the hash inside of it

[worthy cape]

Mine just emits plain boring strings right now, but I'd be tempted to have a PathSet of sorts that one could test strings against. Depends really on what operations you want to do.

[simple ravine]

Well, I think I want to create IFile and IDirectory that is "merged" with the VFS

[worthy cape]

You could save a bunch of effort by storing the common directory name and the leaf strings, handy if you want to access just the filename anyway.

Also, you could reuse intermediate FNV state, you don't have to compute it from the start every time.

[simple ravine]

yeah, I think I'll be creating a structure out of it

[worthy cape]

Assuming you've got an incremental FNV1a, you could give your PathString an additional ctor where you can provide such data up-front, or make it conditional and not generated until you need it.

[simple ravine]

ah, yes.. that's a fractional of the cpu time, but good point

[worthy cape]

Of course, this assumes that it's a cost at all.

[simple ravine]

about 10%

[worthy cape]

I should figure out what kind of interface I need from the end-app code and implement accordingly, needs both directory listings and direct full path access.

[simple ravine]

I dont want the consumer of the API to worry about bundles at all, it should be like they're not there

so I will want to provide the inside tree with the rest of it as one

they'll just be different implementations of an interface of IDirectory and IFile ideally

hopefully inside files are not overlapping two different bundle files

[worthy cape]

What do you mean?

Files being both in a bundle and "loose"?

[simple ravine]

yeah

so if you want to access Mods.dat for an example, you'd just do something similar to...

var fs = new Ggpk(pathToGggk)["Data"]["Mods.dat"];

[worthy cape]

I'm a bit afraid of hitting the disk on every directory listing.

[simple ravine]

what's the alternative approach you're thinking of?

[worthy cape]

Works better if you're overloading [] to look for the particular name in a bundle first and then on disk only if it doesn't find it in the bundle set.

[simple ravine]

well, yes it would index the bundle index proactively, and have that in there ready to go

[worthy cape]

While if you do something like generate a full concrete Dict<String, Entry>, you're going to pay the enumeration cost up-front for both index and disk in order to produce a merged dict.

[simple ravine]

not necessarily double the cost each time, assuming most files are bundles, you'd check the bundle index first

[worthy cape]

You can't check if first if you're working with dicts like that tho.

Not sure what kind of type your ["Data"] would operate on, my C# is a bit behind.

[simple ravine]

if you overload the indexer

[worthy cape]

Icky fact - my Steam PoE installation is 133k files across 2353 folders.

[simple ravine]

public class VfsExample
{
    private readonly Dictionary<string, object> _bundleStuff;
    private readonly Dictionary<string, object> _otherStuff;

    public object this[string index]
    {
        get
        {
            if (_bundleStuff.TryGetValue(index, out var bFile))
                return bFile;
            else if (_otherStuff.TryGetValue(index, out var oFile))
                return oFile;

            throw new Exception("boo");
        }
    }
}

simplified example obv

[worthy cape]

(last I looked at the cursed platform, C# didn't have operator overloading 😄 )

[simple ravine]

it has all the overloadings

it has had it for like 10 years or so?

assignment operators are not possible though

[worthy cape]

Given that this exists, for some reason I thought C# didn't have overloading but it might not just have been sufficiently capable for something like Spirit's metaprogramming.

https://www.codeproject.com/Articles/5676/Spart-a-parser-generator-framework-100-C

[simple ravine]

I'm not familiar with Spirit's metaprogramming

[worthy cape]

Spirit uses a bajillion overloaded operators to express PEG grammars succinctly as a DSL in C++.

Anyway, yay, make something cool.

[simple ravine]

🙂 hopefully

[distant rock]

Any news on when/if the character-window API will be available through OAuth? The newer chrome versions with SameSite props scare me a bit for the future of Exilence.

(Not entirely sure what that property will implicate for us yet)

[simple ravine]

Are Electron apps affected by SameSite at all?

[distant rock]

I mean they should be as they are just wrapping chrome, so I dont see why not, even though you could probably get around it somehow

But I just upgraded to Electron 10 locally and getting this rn:

[simple ravine]

guess you'll have to ship it with a backend

[frank kayak]

i also had CORS problems with queries from wasm to the poe api

[civic crane]

@distant rock https://github.com/SnosMe/awakened-poe-trade/blob/master/src/main/cf-protection.ts you can modify response in main process

[distant rock]

Interesting

Thanks a lot dude, will take a look

[frank kayak]

thats probably only possible in electron?

[worthy cape]

Upside with freestanding Electron is that you have more control over the protections that a browser would have as mandatory.

Until the underlying engine stops exposing things like sameSite overrides.

[distant rock]

yup

yea you cant even auth using the session id in a web browser since it only allows requests from the same site @frank kayak

[frank kayak]

noticed that

im not into web dev but when i noticed that i can compile for wasm target with not too much effort i had to try

did disable cors temporarily for dev as it should be possible inside a .webextension

[worthy cape]

I find it amusing and sad whenever people claim that CORS and similar protections are "broken" as a freestanding agent can do whatever it wants.

Completely missing who it's supposed to be protecting.

[frank kayak]

i messed up with samesite and cors thought they were related but it was the Access-Control-Allow-Origin stuff...

cors is good

[simple ravine]

Same with people who say OAuth is broken, because one or two implementations had a security vulnerability

[frank kayak]

its just annoying for me when im trying to figure out how to make sth work

@distant rock are you an author of exilence?

[distant rock]

Yea @frank kayak

[frank kayak]

i think i read a comment before

ah ok

already thought i was imagining it because the search didnt find anything

[distant rock]

@civic crane So it should be fine until the next electron update if im understanding it right?

Ah

😄

[frank kayak]

still pretty tired from yesterday...

[civic crane]

@distant rock I don't think it will ever break, but just a reminder to check this on electron (=chromium) version bump 🙂

[distant rock]

ah thanks, gotcha @civic crane 🙂

[worthy cape]

@civic crane Yeah, there's a lot of weirdness, like the Art you mentioned that doesn't encompass its children - https://zao.se/~zao/poe/debug/6498111448428023683-tree.html

Feels like recursive_size has a bit of overloaded meaning but I can't quite discern what.

(finally got around to write some exploratory code)

[civic crane]

I think they kinda stitching file indices from multiple systems at build time, and don't care about this edge-case on a root level folder (Art & cachedshaders). I'm more worried about hash mismatch for Art/Models/Effects/monster_effects/League_Heist/military/melee/1h_swordShield

[worthy cape]

Yeah, that one looks odd

I wonder if the file Art/.dds is supposed to be that way or some misinterpretation from our side

[civic crane]

yeah content of Art is strange. I think more and more that Art as a root folder is left for compatibility

[worthy cape]

Heh, this partial code compiled and surprised me a bit by continuing execution ^_^

            std::vector<char> 
            exit(1);

[simple ravine]

is it possible to just iterate with that command algorithm for path building using recursive instead perhaps?

[worthy cape]

Not quite sure what you mean?

[simple ravine]

When I map filenames, I do it over the Payload Size and not the Recursive Payload Size

anyway; it's probably a bad idea.

[worthy cape]

Payload size is always consistent, the three anomalies we have right now is that Art has the same own size and recursive size; and two intermediary entries have mismatching hashes: https://discordapp.com/channels/174993814845521922/175290321695932416/769286372305141783

[simple ravine]

another interesting thing is that the path bundle records are not ordered by offset for some reason

[worthy cape]

Yeah, that's odd.

[worthy cape]

Hrm, E87816F6BAF40000 - Art/Models/Effects/monster_effects/League_Heist/military/melee/1h_swordShield matches my table in manifest 6498111448428023683.

[civic crane]

table in manifest?

[worthy cape]

https://gist.github.com/zao/cdc48f333e6bda8715e103d76e01af40

There's a matching "empty" entry for that hash in the path_rep set of the patch before current.

[simple ravine]

wat

[civic crane]

ahh yep, that's correct folder must be empty, but in current index this empty folder is missing

[worthy cape]

It's present in the current Steam release as well. Are you working off Standalone or Steam?

[civic crane]

Standalone from cdn

[simple ravine]

would have been nice if they were sorted in some kind of sane folder structure

like Art .. Art/Subfolder1 ... Art/Subfolder2 .. etc

[worthy cape]

There's two possibilities - either there's a secret order or there's no intended order.

[simple ravine]

well, yes

[worthy cape]

As in: "let's iterate this hashmap linearly"

One could look at the table between patches and see if it changes in any interesting way.

[simple ravine]

trying to work out a good way to make a graph structure out of this

[worthy cape]

@civic crane I find "Art/Models/Effects/monster_effects/League_Heist/military/melee/1h_swordShield"'s hash successfully in the current Standalone GGPK as well.

1548121 1548129 1548439 8 318 E87816F6BAF40000 <empty>

[civic crane]

wtf is going in my code then 😅 will save binary and search it in sublime

[worthy cape]

It's not a recent change either in the Steam client, it's been present since the tech patch.

[simple ravine]

Art/2DArt/BuffIcons/4K/Beserk.dds nice typo 😄

guessing it should be "Berserk.dds"?

[worthy cape]

There’s a lot of historical typos all over, and a lot of internal names.

[worthy cape]

Like say, the affliction (delirium) effects “sanic” and “slamma_slam”

[frank kayak]

-> "Referer" 😛

[simple ravine]

I guess technically correct

haha, yes 110011

[frank kayak]

oh it even exists...

[worthy cape]

Some things are NZ spellings too, people get quite upset some times over “blatant typos” in skill texts.

Focussed comes to mind

[simple ravine]

yes, some derivate of british

[golden bane]

@worthy cape Regarding Art/.dds: That path was also present in the _.index.txt in the first bundle update, so it's likely an error on GGG's side

[civic crane]

00 00 F4 BA F6 16 78 E8 as I thought, was suspicious of this zero bytes

funny that this code actually worked for most cases

[simple ravine]

oof that flashbang

[civic crane]

btw there are 26 dirs with this warning

[worthy cape]

8907143    8907313    8907321    170    178    F0840EA8F0F58E62    Metadata/Items/Gems
8907313    8907321    8907321    8    8    68AF22CC78B62D6F    <empty>
8912940    8912998    8913006    58    66    F2011F2D9A12890D    Metadata/Items/Maps
8912998    8913006    8913006    8    8    BA268D14D8D991C6    <empty>

(columns are offset; offset + size; offset + recursive_size; offset; size; hash; name)

Those two seem to have subdirs with unknown names.

[simple ravine]

Any content matching those unknown hashes?

as in files

if not, could that the generating algorithm produces some garbage for some reason

[worthy cape]

I think I checked in the past and there were no collisions between file path hashes and dir path hashes.

(verify if you like 😄 )

These would probably be best found by looking at legacy GGPK trees or brute-force the leaves.

[simple ravine]

Well, there are 6000+ PathBundle where PayloadSize is 8

doesn't matter though, after extracting all file names from the inner bundle, it matches exactly the number in file count

[worthy cape]

Non-leaf ones can be trivially verified by substringing their children, the sneaky ones are the empty leaves.

[simple ravine]

what I do is not touch the recursive size at all at the moment, and just enumerate the content of offset -> offset+payload_size, as they're overlapping anyways

if there are empty leaves, I assume it to be garbage?

perhaps empty folders of some kind, that accidentally got hashes into the path mapping stuff

[civic crane]

you don't need offset+payload_size if you want all file names, just pass pathreps bundle to your method

[simple ravine]

what do you mean?

[worthy cape]

As there's no holes, you can just jam in the whole thing indeed and it alternates phases between building bases and generation of paths.

[simple ravine]

ah

[civic crane]

when using offset+payload_size, base mode occurs only once at start

[simple ravine]

fair point

so what are those seemingly empty ones for then?

[worthy cape]

May be more obvious once their names are found.

Mountpoints or some other aspect of how the data is sourced?

Unused or partial references from data tables or data files, causing the path to be considered "used", but with no members?

Depends really on how the tooling for building bundles and the index is designed.

[simple ravine]

how did u find those empty "leaves"? I made a thing, to find empty ones, but it only found 5787 out of 6058 empty PathBundle things

ah nevermind, where payload_size and recursive_payload_size == 8

this is the most annoying thing I've encountered in quite a while

[swift beacon]

Art/.dds seems to be be a valid file

This is what I get

Semi-transparent texture of some sort

[worthy cape]

Yeah, the question is if the name is actually what's generated, or if it's a glitch.

As the file entry necessarily needs to match the hash it's self-consistent in the index, so I guess it's an authoring mistake.

[swift beacon]

That would be my guess as well

[civic crane]

@swift beacon is your viewer oss?

[swift beacon]

Just have it on my local machine, not in a repository anywhere.

[frank kayak]

could that Art/.dds be part of some logo icon? somehow slightly reminded of one - could be from something else though

[frank kayak]

current logo - seems im quite a bit off with my guess...

[worthy cape]

It's not an exact copy of any other DDS file, at least not according to SHA256.

In any way, I'm more curious as to what the fileless directory entries are and whether I should burn corehours on brute-force.

[frank kayak]

what are those "fileless directory entries"? sorry but i have no clue about the entrails of the ggpk (yet)

[worthy cape]

@frank kayak there are directory entries in the bundle index that generate no file names, just containing nested directories or nothing at all. Some of them can have their name reconstructed by looking at descendant paths and strip away components to get to the correct ancestor path name.

[frank kayak]

that sounds like a pain

[worthy cape]

Some however have no descendants at all or have only empty directories as descendants, their names can only be found by brute force testing of potential child names or by cross reference to older sources of path names.

They don't matter for listing all the files or for extracting any files, thankfully, but they make working with the whole tree a bit more of a bother.

[frank kayak]

could that perhaps be some ugly handling by the ooz lib?

or does that also happen with the official oodle?

[worthy cape]

The structure seems consistent enough

[frank kayak]

ok

at some point ill mess with the file contents too...

sounds fun

[worthy cape]

One notable thing is that if you look at the directory names in inner block storage order, they seem to be alphabetically sorted except for the parentless CachedHLSLShaders/DX11 and CachedHLSLShaders/Vulkan ones at the end.

[worthy cape]

Got a result from the brute-forcer, 99AC90C1BB555484 "Art/Models/Terrain/EndGame/ForkingRiver/tgms"

[swift beacon]

Interesting. It's an empty directory?

[worthy cape]

Yes, it's an empty leaf directory.

3987776    3987928    3988247    152    471    FC75BEF34E7F3F1E    Art/Models/Terrain/EndGame/CowardsTrial/Vaal
3987928    3988056    3988056    128    128    8978F035B41EB036    Art/Models/Terrain/EndGame/CowardsTrial/Vaal/Bridge
3988056    3988247    3988247    191    191    505AD865EA213326    Art/Models/Terrain/EndGame/CowardsTrial/Vaal/tgms
3988247    3988255    4032355    8    44108    B975868600F64E2C    <empty>
3988255    3988800    3988800    545    545    600DB2BF045117FF    Art/Models/Terrain/EndGame/ForkingRiver/Doodads
3988800    3988808    3988808    8    8    99AC90C1BB555484    <empty>
3988808    3988816    4032355    8    43547    6AC8CCE9B6EF036A    <empty>
3988816    3991899    4017145    3083    28329    B0F28C51D055B177    Art/Models/Terrain/EndGame/ForkingRiver/Tiles/AquaductAdaptations

I have now broken the tool and must sleep 😄

[swift beacon]

😴

[simple ravine]

PSA for any .net developers in here who do string comparisons (likely most)... Starting with .NET 5 there's a change in how IndeOf and Contains works.

[worthy cape]

We're PHP and JS now 😛

[grave wren]

what's that 1475 magic number now

[simple ravine]

Faust, just expected index on his test case. Reference: https://github.com/dotnet/runtime/issues/43736

Oh, also different behavior depending on windows version: https://docs.microsoft.com/en-us/dotnet/standard/globalization-localization/globalization-icu

Windows 10 May 2019 Update and later versions include icu.dll as part of the OS, and .NET 5.0 and later versions use ICU by default. When running on Windows, .NET 5.0 and later versions try to load icu.dll and if it's available, uses it for the globalization implementation. If that library can't be found or loaded, such as when running on older versions of Windows, .NET 5.0 and later versions fall back to the NLS-based implementation.

Thankfully you can opt out of the behavior, or ship app-local icu.dll

but it's gonna create some initial ruckus before people learn.

A little annoying though, as cultures and stuff is hard enough as-is

[grave wren]

tbh this is a bad decision

[simple ravine]

It's gonna sting for a little while, but I think the sentiment is well-meant. Make it work the same across platforms. I just don't know why they won't ship the icu stuff with .net instead?

[grave wren]

yeah that'd be the way to go with default staying or being patched up

[worthy cape]

A whole bunch of runtime stuff ships with the OS nowadays, and it's a great thing for deployment.

Probably some cross-pollination from the UCRT?

[simple ravine]

Well, funnily enough they went the other way with windows UI components that was exclusively only available for UWP applications. They are now going to ship those as packages instead. There has been a lot of work going into that being able to happen.

[simple ravine]

Ugh... I am starting to not enjoy how Word and PowerPoint have become my primary tools in my day-to-day things.

[frank kayak]

latex?

[simple ravine]

i prefer cotton pants, thanks. especially when wfh

[frank kayak]

...

[simple ravine]

and nah, need to use word etc, collaboration and all

[frank kayak]

or generating latex with ci

[simple ravine]

i don't think that works well with the real-time editing collaboration stuff in Office 365

[frank kayak]

yeah cotton is in general best

the last word version i used a bit more was prob office 2007. other than outlook i dont use any other office program atm

i did generate xlsx though (in go)

[grave wren]

Why generate latex just use asciidoc

[simple ravine]

I am trying to create a new trend in the job titles, though. Right now we just have 'Cloud (Solution) Architect', but that's not enough. It

[frank kayak]

i dont repeat that mistake...

earth? marine?

[simple ravine]

Well, I see 3 or 4 different niches

[grave wren]

Space cloud architect

[frank kayak]

oh

[simple ravine]

• Software (cloud native, distributed systems, microservices, all that jazz)
• Infrastructure (virtual networks, iam, peering)
• Devops engineer
  (- Security specialist) maybe.. not fully made up my mind

[grave wren]

Why does DevOps engineer need cloud in the title

[simple ravine]

well it doesn't really

I just want to separate the responsibilities, so what I'm proposing now is to have

• Cloud Software Architect
• Cloud Infrasructure Architect
• DevOps Engineer

[frank kayak]

"devops engineer" wasnt devops meant to mean some team mentality?

[simple ravine]

It's obv a bit overlap, but that's just good.

[grave wren]

It was but it's easier to just plop a specialist in @frank kayak

[frank kayak]

yeah...

[mortal bone]

It is a mentality, but it also means the person setting all that up haha

[simple ravine]

Well DevOps Engineer, is like SRE, and deal with setting up and manage pipelines, and ensure build environments work etc

[grave wren]

Heard the term enough when helping others with pipelines

"you'll do DevOps for us" yeah bud not how that works

[simple ravine]

SRE is two things actually lol.

• Site Relibility Engineering
• A Site Reliability Engineer

[frank kayak]

i think we need more trendy words...

[simple ravine]

well, you can bake it into your other architecture/engineer roles of course, or you can have someone specialize in it

both work

[mortal bone]

Yeah, I do our build/release management at work for our team

[grave wren]

If you'd be out for months would anyone be able to do what you do

[simple ravine]

We also have people who specialize in setting up all the pipelines, and such.. it has become a bit bigger, with all the Infra as Code, Pipeline as Code, Configuration as Code etc

You could ask the same with any key role though

So I think it's more of a 'how do you build a resilient organization in general'-quesiton, I think

[grave wren]

If you have one specialist and nobody else can do much if stuff fails that's not DevOps in my books

[simple ravine]

Thats why there are 2 pilots in a plane.

But yes, you have a point Faust, and I think that DevOps should be more spread out

[worthy cape]

We're more into DevOops

[simple ravine]

PagerDuty returns an immediate "Did you try to turn it off and on again?" 😄

Faust, at your org do you have autonomous teams (product teams) or more "here's a bunch of resources, and here's a bunch of projects:"

[grave wren]

We don't do products in theory but it's very autonomous in terms of developing for customers

[simple ravine]

in the latter, it's easier to get away with having a couple of specialized in devops, where you have to be a bit more ninja in an autonomous team, especially if they are exclusive to a single product

[grave wren]

The problem is DevOps aims to be an enhancement for everyone if you just plop in a magical pipeline and say that's DevOps then that's not cutting it

Don't like the term either way cos everyone defines it differently

[simple ravine]

I agree. DevOps is a culture. What I refer to with DevOps engineer is the one who creates the CI/CD glue and manages that part.

But yes, DevOps as a whole is much more than that.

It might not be the right way to set up a team that way everywhere, but at some places it makes sense

[grave wren]

In my small team everyone was able to do it after introducing them to it while I still did the main work. Imo that'd be DevOps what people expect is the opposite. You go in do the grunt work bam done DevOps executed

[simple ravine]

Not sure if I understand what you mean with the second half there

[grave wren]

Let me rephrase: it seems like a lot of people just want a pipeline that works and a person that's available to fix it if anything needs changes they don't want to actually understand or maintain themselves partly

[simple ravine]

Well, that's just the CI/CD part of DevOps, which is important, but far from what we mean with DevOps

[grave wren]

Ofc but if that part is an issue already I am not sure if people really wanna do DevOps

[simple ravine]

Someone needs to ensure that monitoring, incident response, testing is done properly, release cadence works as expected, etc

Well, for me it's not an issue, 🙂 I think DevOps culture / SRE is great. I.e. you build it, you run it etc

But if we're talking in general, where I see the conflict happening is with budgeting and procurement

[grave wren]

Yep definitely as well as general staffing depending on project size

Knowledge sharing is also a thing that's hard across a company

[simple ravine]

As well as making sure you have a best practice in your company, so that project a and project b is somewhat recognizable if you need to shift people around

where technically feasible ofc

[grave wren]

Funnily enough the only company section that does sharing rounds every month is the sap section

Everyone else just does whatever mostly

[simple ravine]

Yeah, that's gonna hurt the bottom line

Faust, you familiar with Google's view on DevOps - SRE?

It has gained a lot of traction among DevOps thinkers the past couple of years.

If not, if you want take a look at https://landing.google.com/sre/ they have a couple of free books on the subject as well. Let me know what you think sometime

[grave wren]

Not yet I am mostly stuck in ci/cd for a while now but I'll take a look

[simple ravine]

This 5 minute video is pretty decent https://www.youtube.com/watch?v=1NF6N2RwVoc

[grave wren]

I feel like that video lacks any meat but I am really not a fan of super fluffy material

[simple ravine]

Right, there are better videos. I'll find the one that made it a bit clearer for me at first

https://www.youtube.com/watch?v=uTEL8Ff1Zvk&list=PLIivdWyY5sqJrKl7D2u-gmis8h9K66qoj

[worthy cape]

One important thing to remember is that you're not Google or Facebook in scale.

[simple ravine]

this playlist are small bites of how they do it at google

Right, and also like to note that SRE started as a very small thing in Google though and I've heard several people who started startups use SRE concepts successfully. It's more about the practices and mindsets

the difference is that in smaller organizations, people wear multiple hats, but in larger it might be necessary to have people specialize for various reasons (limiting context switching, necessity to specialize and get deeper expertise)

Excuse the readability due to the background, but here's DevSecOps at Enterprise scale for modern organizations:

there are a lot of parts in there.

and this is still just from a tech perspective

[simple ravine]

are the pigs flying yet? 😂

[worthy cape]

hash,path
D200409415842B4E,"Metadata/Effects/Spells/monsters_effects/League_Affliction/Affliction_Phys_Demon/epk"
0CB4F6B5CB69910A,"Metadata/Effects/Spells/monsters_effects/League_Affliction/Affliction_Poison_Demon/epk"
328AA4CEC4E8E0CA,"Metadata/Effects/Spells/monsters_effects/League_Incursion/Vaal_Architects/spells/ultimate_spells/lightning_hurricane/hit"
...

Full list thus far at https://gist.github.com/zao/fbbcf073ce404f0dd964f78d1897fc16

I should add concurrency to this and actually persist results 😄

[lean spindle]

Does anyone have any recommendations for decluttering the text in this app at high tiers? Atm I'm considering disabling map node names by default, because I'm not sure how to make this visually appealing/readable https://dev.poeatlas.net/

Any suggestions would be greatly appreciated. (You can fiddle with visual settings in the 'options' menu (top-right))

[simple ravine]

nodesize = 0.7
text size = 0.85

[civic crane]

for me it's better at initial zoom hide names, keep sizes to 1, but when you zoom in show names and apply thesensei's params

[lean spindle]

Ty for the recommendations zen and Snos! I will put your advice to good use.

[frank kayak]

i tried passing 2 errors packaged with fmt.Errorf %w upwards - seems thats not supported: "Errorf call has more than one error-wrapping directive %w"

oops wrong discord

[frank kayak]

better then not finding it

[worthy cape]

Sounds like we'll have a relaxing holiday period this year, heh.

[frank kayak]

are you also getting a lockdown?

[hazy fog]

he's talking about league delay

[frank kayak]

ah didnt follow poe news lately that much

[hazy fog]

I just pinged everyone with it

[worthy cape]

What's up with the duplicated omegalul reactions by the way? Didn't get to add a friendly one 🙂

[frank kayak]

have all channels but tooldev muted

[hazy fog]

lots of servers have omegaluls zao

[worthy cape]

Ah.

Upside of things, maybe I'll be visited by the productivity fairy and get some code done.

[frank kayak]

didnt i smash that fly recently?

[worthy cape]

Seems so, all I've done this week is noodle around on how to accelerate bruteforcing.

[frank kayak]

and i enhanced a cookie lib...

[hazy fog]

if all you do is hang out in tooldev why haven't you made soemthing to get a tooldev role

[frank kayak]

huh?

[frank kayak]

nothing public yet

[hazy fog]

that's the only requirement

[frank kayak]

zao currently the only one who has access to my mess - i want to clean it up a bit more

its (will be) here: https://gitlab.com/poem-dev/poem

[hazy fog]

if zao says it exists I can still grant the role

I trust him

[worthy cape]

@hazy fog It compiles, it does stuff with the API, looks useful.

[frank kayak]

wouldnt be so sure...

😛

[worthy cape]

It compiled last I touched it, Schrödinger would be proud.

CLI and GUI tool to query assorted endpoints in a friendly fashion.

[hazy fog]

Making an exception because I can. @frank kayak is working on this https://gitlab.com/poem-dev/poem with vouch from @worthy cape he gets tooldev because I am a benevolent dictator

[frank kayak]

hah ok - ill accept...

[hazy fog]

I like to be able to differentiate the idiots asking dumb questions from the people working on stuff at a glance

[frank kayak]

you mightve done a mistake...

[hazy fog]

doubt it

[frank kayak]

if anyone wants to look at it i can grant access on gitlab

[hazy fog]

[frank kayak]

@worthy cape you did mess a bit with go right? do you have a clue about doc examples?

https://golang.org/src/sort/example_search_test.go i think im doing that weird naming wrong

[dull laurel]

has anyone ever thought about making a github repo with issues (or a similar approach) for all the QoL features and stuff like that mentioned on reddit, the forum and elsewhere? Like a big public list of todos (or wishes) similar to uservoice

[frank kayak]

i have a collection with copy paste stuff under ~/d/

calling it with dcat(){ cat ~/d/"$1"; }

for example a progress bar in bash for i in {1..200}; do printf '%s\033[s\033[G%0.3d \033[u' . $i; sleep .03; done

and other useless stuff

[dull laurel]

if that is reply to my question, then I might need to correct the question. I meant Path of exile features

[worthy cape]

@frank kayak No clue, sorry. Documentation isn't my forte.

[frank kayak]

i did see some tries at doc collection but its mostly spread out

ok thx :/

i iterate through the possibilities no luck yet

[worthy cape]

Note that the filename is significant and needs to end in _test.go

[simple ravine]

on procrastination and productivity... all I have achieved this week is write two job descriptions in 1 word document and grab some files and send it to a colleague.

i have no idea whats up with me laterly

[frank kayak]

@worthy cape yes and its good to add _test to the package name too so that the stuff from the package has the package.-prefix for the users.

solved it now

problem was the case

its Example_label() and not Example_Label() for package examples

[worthy cape]

Got to love magic naming.

[frank kayak]

yeah...

the public/private with the initial letter is really nice though

go will become a bit faster with 1.16 https://www.infoq.com/news/2020/08/go-register-calling-convention/ 🙂

[swift beacon]

@hazy fog do screenshots count? 😂

https://imgur.com/a/qvdRmmW

[hazy fog]

[worthy cape]

go at man bank

[dull laurel]

in browser with webasm?

[hazy fog]

zao what's your ruling

I dep[utize you

I spend way too little time here recently

[swift beacon]

It's supposed to be an album but discord is only showing the first image for me...

[worthy cape]

If you write text before an imgur gallery link it shows the URL

(the header still links to the album but may be less obvious too)

[frank kayak]

@worthy cape isnt discord in rust? and no longer go? - def. wouldnt have happened with go...

[worthy cape]

@frank kayak imgur is having serious server problems the last few days

[dull laurel]

isn't discord electron?

[worthy cape]

https://status.imgur.com/

[frank kayak]

https://blog.discord.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f

@swift beacon what languge are you using for your project?

[worthy cape]

@swift beacon Meshes look good - I guess you're not doing materials or skinning yet?

[swift beacon]

C#

[dull laurel]

wow, that is impressive

[frank kayak]

@dull laurel i meant serverside - the standalone is electron yes

[dull laurel]

@swift beacon is that a webapp?

[swift beacon]

I'm converting the models to .obj + .mtl

[worthy cape]

Ah right, that rings a bell.

[swift beacon]

@dull laurel Yes

[hazy fog]

😺 its official

[frank kayak]

looks interesting

[dull laurel]

that was posted half an hour ago

[swift beacon]

Thanks

[worthy cape]

Best thing about @swift beacon doing all these assorted viewers is that I don't have to 🙂

[hazy fog]

hmm

maybe its too dark

[swift beacon]

I'm basically trying to be able to preview as many file types as I can

[dull laurel]

never let developers do UI/UX 😄

[simple ravine]

well there

[frank kayak]

zao vanished

[simple ravine]

scary

[frank kayak]

alpha -> ff?

[swift beacon]

Have most of them except for the terrain 3D models

And yes, the UI is pretty terrible 😂

[frank kayak]

nah

[hazy fog]

@worthy cape try that

[frank kayak]

you havent seen my tries

[worthy cape]

beep boop

[hazy fog]

better

[simple ravine]

cool, whatever it means 🙂

[worthy cape]

@swift beacon Did you do ARM yet? It's super fun - https://i.imgur.com/Qc90zP8.png

[hazy fog]

it means I will defer to him if he thinks someone should have the role because I cba to research everyone who should

[swift beacon]

Just displaying them as text for now

Looks interesting

[worthy cape]

Got the overall structure of the format parsed, but the meaning of the numbers in the grid is largely undiscovered.

@hazy fog Based on the work of and discussions with @swift beacon , I feel that they're a productive member of this channel and worthy of the green adornment.

[hazy fog]

shazam

[swift beacon]

Yay, I'm green!

Thanks guys

[hazy fog]

[simple ravine]

we the gremlin's of spooktober 😄

[worthy cape]

I've got my hash unveils thus far here by the way - https://gist.github.com/zao/fbbcf073ce404f0dd964f78d1897fc16

Odd mix of old and new content, I wonder if there's a redirect/symlink system somewhere we're missing, or if it's just artifacts of moved data.

I don't have any easy access to legacy trees at the moment, would be interesting to see if any of those were populated before the tech patch.

[simple ravine]

the paths looks fairly "stray"

would be interesting to understand what "epk" and "tgms" means though

[worthy cape]

TGM is a file format

Seems that I only just got started on TGM and TDT in my Rust implementation.

[swift beacon]

I think epk is for effects and tgm is for terrain

[worthy cape]

EPK is some sort of vfx description, IIRC.

[simple ravine]

ah

[frank kayak]

discovered some more black magic with the godoc tool - i need something outside the example func, a var or func so that the whole example file gets displayed...

[worthy cape]

@simple ravine Some of these have rather deep nesting too, some are four levels of unknown - https://gist.github.com/zao/38391b220442b71439ec49da74350d03

Like the children of Art/Models/MONSTERS/GenericBiped/BipedMedium/ModelVariants/HeistRobot.

[simple ravine]

oh that's quite strange

[worthy cape]

Could of course have been moved elsewhere.

[simple ravine]

I am getting curious on how these references are generated, and why there are residuals like this

[frank kayak]

i would guess we probably are missing some important info

zao you ran a hash func over all known paths and compared it to not connected ones?

[worthy cape]

Computed the partial hash of each known parent, then generate all 1-char, 2-char, 3-char, etc. combinations and compute the tail of the hash and check it against the known hash.

[frank kayak]

oh

[worthy cape]

Implementation is stupid tho, doesn't reuse any generation effort between different paths and isn't threaded.

[frank kayak]

cant imagine the game doing something similar

[worthy cape]

But yeah, testing base/#, base/## with an alphabet of " &'()+,-./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ[]_abcdefghijklmnopqrstuvwxyz"

(scientifically obtained by looking at what characters were present in all the generated file paths)

75^N possibilities <_<

[frank kayak]

perhaps a whitelist for the patchserver files?

[worthy cape]

After around 1.5 days on one thread it's gotten to the middle of the 6-letter strings.

Does the patcher know anything about bundles tho?

Or is it sucking down the index and bundles blind as if they were regular files?

[frank kayak]

no clue - not trying to look into ggpk and that in addition to all the crap i already started

has to wait

[worthy cape]

Spreading yourself too thin? I'd never 😄

[frank kayak]

i prefer wasting far too much time on the most useless parts...

[dull laurel]

documentation and such? 😉

[frank kayak]

doc never.. at least normally - currently i am trying to add doc to some foreign repo

and learning all the nice quirks of godoc

[worthy cape]

Hrm, I don't seem to be able to get past cloudflare for steamdb.

[frank kayak]

is that a webpage?

[worthy cape]

https://steamdb.info/depot/238961/manifests/

[frank kayak]

why not view in the browser and copy paste the curl command?

[worthy cape]

Works in Edge, Firefox seems upset.

[frank kayak]

not mine

[worthy cape]

I'm running Beta, 83.0b2

[frank kayak]

but google often gave me captchas for regular searches...

always lts on debian (78)

[dull laurel]

isn't there a library for that, i mean to download steam packages? at least in c# there is one

[worthy cape]

It's complaining about sameSite cookies and cookies that have already expired. Ugh.

@dull laurel Yes, there's SteamKit and DepotDownloader. I use the website to look at what manifests there are, haven't been arsed working with the API when SteamDB aggregates it for me already.

[frank kayak]

curl 'https://steamdb.info/depot/238961/manifests/' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' --compressed -H 'Connection: keep-alive' -H 'Cookie: __cfduid=...; cf_chl_1=...; cf_chl_prog=...; cf_clearance=...' -H 'Upgrade-Insecure-Requests: 1' -H 'TE: Trailers'
``` has this format

[dull laurel]

ah, now I understand. I thought it was a lib for steamdb, but its for actual steam.

[frank kayak]

gl with cloudflare...

[simple ravine]

There used to be a library for getting past Cloudflare thing as well

They seem to be working less ideally nowadays after looking at them though

[worthy cape]

I would kind of assume it to work with an actual browser with a meatbag at the keyboard 🙂

[frank kayak]

if you copy the browser behaviour well enough it mostly worked for me - sometimes a few more cookies were necessary or the user-agent

[simple ravine]

I think it also depends on how the site owner has configured it?

[frank kayak]

i like cloudflare

...they use go https://blog.cloudflare.com/graceful-upgrades-in-go/

@worthy cape chromium?

[worthy cape]

Well, I'm not automating anything, I just want to look at the site 😄

[frank kayak]

does it work in another browser?

[worthy cape]

Yeah, Chromium-infested Edge is fine.

[frank kayak]

did you mess up in about:config or the wrong webextension

eh

i did get the cookies out of the old edge

and most other browsers...

index.dat and cookies4.dat are left

[worthy cape]

Maybe it's also related to being on v6 and thus being a Suspicious Individual.

[frank kayak]

v6?

you could try a blank firefox profile

if it works there you messed up your profile

firefox -new-instance -P for the profile manager

[worthy cape]

Might have done something, hard to tell when the difference between profiles is everything.

Especially when it's something as fickle and external as CF validation.

[frank kayak]

yeah

are filters applied to objects or onto them?

[worthy cape]

Installed Windows 20H2... how convenient for them: https://i.imgur.com/q51y0G4.png

All the file associations that Edge could happen to handle conveniently were corrupted.

[frank drift]

think the same thing happened to me

[worthy cape]

Did it this spring for 20H1 too, would be nice if it just wouldn't.

[frank drift]

it also launched Edge fullscreen at startup and asked me to make it the default browser

so much for United States v. Microsoft Corp

[worthy cape]

Now this isn't a problem I expected after upgrading Windows...

C:\PROGRA~2\MICROS~1\2019\COMMUN~1\VC\Tools\MSVC\1427~1.291\bin\Hostx64\x64\cl.exe <<<snip>>> -c ..\..\..\apps\prober\prober_main.cc
  CreateProcess failed: The system cannot find the file specified.

The Ninja build files from VC++'s CMake functionality uses 8.3 short filenames for some reason, and the Windows upgrade process doesn't preserve those for installed software.

f.ex it's now C:\PROGRA~2\MICROS~4\...

[simple ravine]

I just realized I have tomorrow off. Perhaps I have some energy and will to finally figure out how the Bundle implementation in PoeSharp should manifest itself.

@worthy cape I thought that those were alphabetically sorted somehow. Weird that they'd change.

[worthy cape]

@simple ravine I assume they're regenerated if a collision happens and things seem to have been created in a different order in the old fresh installation.

2020-09-27  15:36    <DIR>          MICROS~1     Microsoft
2020-09-26  18:47    <DIR>          MIF5BA~1     Microsoft Office
2020-09-23  19:34    <DIR>          MICROS~2     Microsoft SDKs
2020-07-10  00:42    <DIR>          MICROS~3     Microsoft SQL Server
2020-10-18  14:37    <DIR>          MICROS~4     Microsoft Visual Studio
2020-10-29  10:17    <DIR>                       Microsoft.NET

Got to love the Office short-name too 😄

[simple ravine]

makes total sense lol /s

[worthy cape]

The ultimate crime here is probably the good old 260 character limit on paths.

I wonder if CMake behaves if you toggle the Windows knob for longer paths.

[simple ravine]

Didn't recent versions of Windows fix that?

[worthy cape]

Optional knob still as it has the possibility of breaking things.

Python installer asks you if you want to enable it.

You don't fuck with billions of pieces of software that uses MAX_PATH.

I already have it enabled it seems, so no go.

[simple ravine]

well, in the age of node_modules

[civic crane]

they are flat (in most cases)

[worthy cape]

Oh fuck meeee... I have some code I need to audit for range-for misuse... http://josuttis.com/download/std/D2012R0_fix_rangebasedfor_201029.pdf

TL;DR - the lifetime of results of the right hand side of a range-for is shorter than one might think.

for (auto n : blargh().f()) is almost always wrong

[civic crane]

hope compiler will be clever enough not to create lambda if it doesn't see expression on right side

[worthy cape]

The intent is for it to be "similar to" but still good.

[grave wren]

whoa zao got another green

[civic crane]

i see it

[digital parrot]

damn it. Is it possible to fetch json from https://www.pathofexile.com/api/trade/exchange/Heist using cURL ? I am trying, posting the same as in the browser but for some reason I get 400 status-code responses.
Is it possible at all? How?

[worthy cape]

@digital parrot If I grab the query from Firefox's network tab as POSIX curl, it works on my Linux box.

But if I grab the "Windows" flavour, it doesn't in cmd.exe curl.

POSIX one works in Git Bash, f.ex.

How are you constructing your request?

[digital parrot]

Hm. damn it. I start all over again. Almost reproduced in Guzzle/PHP but I guess I missed some little thing.

[simple ravine]

the devil is in the details, especially for computers

semicolons and whatnot 🙂

[frank kayak]

microsofts curl is castrated iirc 😛

[worthy cape]

Microsoft's curl bails out as it has no compression support.

The test above was with haxx-built up-to-date curl.

[frank kayak]

what other important featutes where also stripped? net connections?

[worthy cape]

(none of you saw the POESESSID in the deleted link above <_<)

[civic crane]

im in

[worthy cape]

Already regenerated it 😄

[frank kayak]

@worthy cape poem-tmp poe -i

windows version might need an update

[worthy cape]

Handy. I just did interactively with the browser.

[frank kayak]

after my latest work itll even work with NCSA Mosaic

...joking - only netscape, elinks and co

[worthy cape]

One of my fears about streaming development is the amount of non-public information that may show.

[frank kayak]

yeah

not sure how much a different user account, different ip helps

ill take a short nap

[minor charm]

Is streaming development popular? Last time I checked guys that were streaming development they were on 0 viewers

I was looking at things I was interested in which might be why they were on 0 viewers.

[simple ravine]

There are streamers who have quite some viewers

https://www.twitch.tv/csharpfritz this one

[worthy cape]

I don’t really care for the viewer count or sustainability of it, I just get questions sometimes of how one works on something like this.

[minor charm]

his setup looks alright. I'd probably spin up a proper vm for streaming and have obs locked to the vm, vscode and browser all running within the vm

all secrets kept specific to the user in the vm

[worthy cape]

There’s too much about my personal life intertwined with my desktop, there would need to be a serious adjustment to have an env suitable to broadcast.

Can’t really do meaningful graphics dev in a VM ^_^

[minor charm]

true

[compact isle]

I used to stream mod development on twitch. Not many viewers but it's great to rubber-duck to haha

[minor charm]

rubber duck ?

[compact isle]

explaining your problem out loud usually helps to figure out how to solve it

https://en.wikipedia.org/wiki/Rubber_duck_debugging

[minor charm]

Oh yeah, that makes sense

[simple ravine]

I've resorted to ordering duck from the asian restaurant and eat it while i talk to it

two birds one stone

[rapid pagoda]

coincidentally, that's also the name of the dish. avoid eating the stone.

[minor charm]

If you go to chuanhsing's swagger poe api and export the client sdk. It has a object sanitization class for sanitizing data. Is that a good example of sanitizing for API client's or is it over the top and doing too much stuff?

link is in the pins

[simple ravine]

that's not really sanitizing, just deserializing into something typed

[worthy cape]

@frank kayak Reading a LWN article on Debian's trouble packaging Kubernetes (being written in unholy Go). At some point in time they tried to unvendor all dependencies into distinct packages.

Also some concerns about the short (one year) security maintenance upstream has, making it impossible to do anything but ship a new version when security fixes happen.

Happy I'm not them 😄

[simple ravine]

I love the discussions on Kubernetes. Especially when speaking with clients and their stance on K8s is "It's a silver bullet for all our needs. It will make us cloud agnostic, or not even need to think about cloud at all. Just add some OpenShift and call it a day"

People underestimate the complexity of that beast lol.

[worthy cape]

(can't link the article on, it's a subscriber-only one I was granted a link for)

[simple ravine]

What's the gist of it?

[worthy cape]

Unvendoring the over 200 dependencies is a massive task, Go's insistence on static build and their weird-ass build setup makes it hard to separate at all, and it's a security bomb.

[simple ravine]

I wonder how cloud vendors are dealing with this, like AKS, EKS, GCP's version whatever that's called

Fortunately people who use those have less of a headache, but nonetheless the underlying problem exist, and I think it's a ticking timebomb

[dull laurel]

they update slowly? at least from what i heard from our infrastructure team that was hellbend on setting up their own cluster because azure didn't deliver all the weird features they wanted

[grave wren]

(is pray to the machine god a valid answer?)

[worthy cape]

Always.

[simple ravine]

I've never spun up my own K8s cluster and maintained it. How does the upgrade look like? Do you have to drain the cluster and upgrade, or can you do some kind of in-place upgrade?

Would tools like OpenShift help with that kind of headache?

I mean it looks like k8s is shipping new versions quite often

So if the upgrade procedures aren't too scary and cumbersome...

[frank kayak]

@worthy cape https://lwn.net/Articles/806230/ this one?

[worthy cape]

Nope, 835599 from today.

[frank kayak]

not sure if it was kubernetes but there was a large go project that was written like you would write java code

ok ill take a look

[simple ravine]

How much $ does it cost to get access?

[frank kayak]

"Subscription required"

😛

[worthy cape]

LWN posts are sub-only for a period or time or something, with the ability to generate links to share with a small set of friends.

(one week)

[frank kayak]

i admire the debian devs a lot for all the work they do

[simple ravine]

I don't get the thought around gating articles that way, but whatever I guess

[frank kayak]

i think docker also made a lot of problems initially and then was repackaged as a blob of things in debian iirc

with qt the separation in a thousand packages did annoy me a lot

didnt manage to compile acquisition at first because of some qt web package in a version that wasnt in stable...

[worthy cape]

Reading the FAQ, the distribution of the links can be reasonably wide as long as it's not put into a system to defeat it.

Behave and read https://lwn.net/SubscriberLink/835599/58edea1e7066373f/

[frank kayak]

... thx

[simple ravine]

How do you become a 'subscriber' though? I mean if it's privileged content, I'm happy to do it the right way

[frank kayak]

im still using your POESESSID btw

[worthy cape]

@simple ravine https://lwn.net/subscribe/Info

Depending on the level you also get rid of ads and stuff.

The links are a way of enabling you to show interesting content to others and to hook some into subscribing 😄

[simple ravine]

hah

ads don't annoy me, they are filtered away anyways

(because a lot of them misbehave, the nice ones gets filtered too)

[worthy cape]

im still using your POESESSID btw
@frank kayak I hope you're joking, it's not accepted by the APIs here.

[frank kayak]

i obviously am - sry that i didnt make it clear

somehow missing the systemd "discussion" in the debian comm

[simple ravine]

hehe I like how people still use the "X considered harmful" mantra

so unoriginal

[frank kayak]

systemd is a piece of shit

but its the pi... we use

i cursed a lot about it...

[simple ravine]

Kubernetes vendoring considered harmful
from the article

[frank kayak]

at least it can print qr codes

[simple ravine]

On the discussion of vendoring vs centrally installed libraries in the operating system, I have seen where central library stuff leads us. Just look at .NET Framework and the GAC and all the problems that lead to.

There's a reason why Microsoft abandoned that idea a few years ago. It lead to them having to always think about backwards compatibility, not only in API surface, but also in behavior, or a lot of software could break when you decide to update the .NET Framework on the operating system.

Hence innovation and improvements were slowly grinding slower.

[frank kayak]

https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf the cause

[simple ravine]

I think vendoring is almost always the case

110011, yes the thing I was referring to. It has since then been used, almost at a meme level to use the "X considered harmful"

Funny though, Djikstra's original title was something else. "The case against go to statements", but an editor later changed the title when publishing it

example: "OO considered harmful"

[frank kayak]

the dll mess in windows was never supervised on windows you just install stuff and hope its the correct system dll if it requires it or drop the correct one besides the exe - in linux distros the sos are versioned

didnt know

with go though its pretty different

the deps are just stuffed into a static binary

the dds tried to get go projects separated in dyn linked ones

[simple ravine]

well, if you can have several versions of let's say curl library installed, the case to manage that centrally is at least somewhat diminished

the pragmatist in me just think it's not worth the effort and headache in modern day computing

[frank kayak]

mostly useless...

i think this time i got all

[worthy cape]

I like the naive comments that assume that developers care about SONAME and ABI compatibility these days.

Seems like they've missed the whole semver race.

[simple ravine]

Well, semver is just part of the solution to a problem in an ecosystem where a lot of stuff depends on your code

[worthy cape]

In the package management system I maintain at $dayjob, we go for exact version matches. Nobody got time to figure out what ABI/API guarantees software has (hint: fuck-all).

This is a field where Anaconda is considered a good thing (by users).

[frank kayak]

eh conda

that gigantic ....

did you try static compilation with nuitka? last time i checked it still required conda at a set path

[simple ravine]

Let's take libcurl as an example. It's a pretty good example where you want to have the ability to innovate new functionality, while also be able to provide important security fixes in lower major versions

It becomes a maintenance hell

[frank kayak]

libcurl3 seems to be just an empty relic

looked at the wrong pkg

its redirected at v4

[simple ravine]

Isn't that bad?

[frank kayak]

not when its compatible

[simple ravine]

Aren't there any breaking changes between v3 and v4? Even if there are no API surface changes, there are likely behavioral changes

Otherwise, what's the point of the version change at all?

[frank kayak]

it takes ages till sth lands in debian stable its not like windows where the customer is there for testing

[simple ravine]

lol

Let's put the windows/linux discussion aside though, the discussion of versioning and vendoring, or not to vendor is a pretty big one of its own

[frank kayak]

i think debian has to make compromises for stuff like browsers and go...

[simple ravine]

for any quick-moving things I'd say

[frank kayak]

btw alias xargs='xargs ' and alias sudo='sudo ' are so nice

its just not maintainable with the amount of deps just growing and changign so fast

with those aliasses you can have aliasses after xargs or sudo like sudo ll ...

[dull laurel]

so kubernetes is just one big fat binary?

[simple ravine]

alias please = sudo 😄

[frank kayak]

that reminds me my sudo insults me

you need that "Defaults insults" in the sudoers

if you want to repeat the last command with sudo type sudo !! (history expansion) i use !$ for the last word often

cant wait for //go:embed for no more hex blobs in source code https://go.googlesource.com/proposal/+/master/design/draft-embed.md

its already implemented in gotip but release is in february...

with this we will see more and even larger go binary blobs!

[worthy cape]

Any decade now: https://thephd.github.io/vendor/future_cxx/papers/d1040.html

C++ P1040R6 - std::embed

[frank kayak]

wow

i messed only a bit with c++ around 2012 ( implemented an xpath parser partially naively)

c++ did change a lot but all those additions scare me off a bit

@worthy cape how well do transitions from older c++ traditions to newer ones happen in larger projects? probably a bit messy?

[worthy cape]

There's kind of two schools of thought:
a) "there's versions of C++?"
b) "this codebase requires Clang-trunk"

In the game industry where you want to minimize external factors, it's not uncommon to lock the toolchain down at the start of a project, checking in compilers and dependencies into version control.

[frank kayak]

makes sense

toolchain no longer working would be ...

[worthy cape]

The standard you're on is the one common to all the platforms you target, and changes are slow.

You also have the problem of that a lot of people are not great at the language, being more practical and pragmatic.

That varies on a per-team basis but tends to lean toward KISS. There's been recent movement to adopting some C++ features, but for the longest while there was no interest at all in standard library containers and algorithms, having your things be value types, etc.

C with classes 🙂

[frank kayak]

far from c...

go happened because of c++ despise by ken - originator of b (predecessor of c)

i feared a bit that id find a lot of deprecated features are sprinkled around in c++ codebases where you dont have to many requirements for the toolchain

[worthy cape]

In the scientific field, it varies greatly on the poor PhD student that wrote the codebase but varies between non-standard C++03 to C++14 or 17.

Out in reality with regular business dev, I have no idea but assume they're slow to adapt.

I've heard tales of horrible toolchains from telecom.

Mate of mine just recently got permission to use shared_ptr.

[frank kayak]

hm

i prob will try to enhance my c++ knowledge at some point but stay away where possible...

[primal schooner]

Anybody know who devs Craftofexile.com ?

[frank kayak]

i think that dev is active in the tft channel?

[worthy cape]

There's a discord (just TFT it seems) and a contact email.

[frank kayak]

https://discord.com/channels/645607528297922560/680796887259021342

[worthy cape]

cf. https://www.craftofexile.com/about

[frank kayak]

source code is unpublished?

[worthy cape]

On Anaconda btw - this just in: https://twitter.com/walkingrandomly/status/1322238796310568961

Mike Croucher (@walkingrandomly)

New development from Anaconda.

If you use Anaconda’s repositories in a for-profit or government organization with over 200 employees, you need to upgrade from free to Commercial edition.

Universities etc still have free usage. https://t.co/AyqyAZUKKL

Twitter · Oct 30, 2020, 6:07 PM

[frank kayak]

was it gpl-compatible before?

[grave wren]

tbh i only used conda once because i needed to get a lib running that i couldnt self compile

other than that i am not a huge fan of it

[frank kayak]

ditto

[grave wren]

but its super weird how they just at random do that

[frank kayak]

somehow reminded me of the redis license stuff https://techcrunch.com/2019/02/21/redis-labs-changes-its-open-source-license-again/

[dull laurel]

they have a fair point

we have this "FOSS" agenda in the company I work for and nowhere is mentioned "contribute back", only saving money by not paying license fees anymore. the same at the company I worked before. saved a million or so and didn't give a cent back to the community they benefited from (I think it was R or another framework for rule engines)

[frank kayak]

i think its quite simple if no contributions flow back in form of code or money other support projects often just die - if a company heavily relies on it and doesnt support it that might later be self-damaging

[dull laurel]

but R or redis or anaconda are not small projects

[frank drift]

the people who made the decision probably already got their bonus and are in to the next thing

[frank kayak]

isnt R a GNU project??

i dont really care if i dont like a projects license ill go on a search to find the next worse project

[worthy cape]

The milking of open source software for profit is a widespread "problem", but I'm not sure that these "oh yeah, it's open source but not for you" kind of hackfixes work.

[dull laurel]

can't wait for angular doing that 😄

[golden bane]

@grave wren On not having to compile libraries yourself: https://www.lfd.uci.edu/~gohlke/pythonlibs/ is a gold mine

[grave wren]

Used my mac and needed a grib lib

https://confluence.ecmwf.int/display/ECC

[simple ravine]

I agree with @worthy cape to some degree, the "hackfix" is a reaction to a problem, but I am not sure it addresses the cause, or rather, changes the approach

FOSS for me, is about positive social impact, perhaps not so much "free software", but rather what the software can do in turn to provide positive social impact. If other people use it for profit, likely does not impact the FOSS library/software negatively.

They're just forcing the consuming enterprise to take a stance, and potentially provide some cash.

Sue the cash might help, but it won't help in the long run, as their opinion on the project will likely change, and they might be looking to replace it either with in-house or other foss that hasn't gone that route at some point.

FOSS that become very popular should rather, look at two things:

1. Commercial support in bounties and consulting. It can be quite lucrative.
2. Introspect and make sure contribution is as frictionless as possible, ensuring that once an enterprise or individual wants to contribute, that it's a welcoming and open experience - otherwise you're doomed to maintain it yourself

[minor charm]

The toolchain we supply to customers is whatever they want for their c++ app. Or if none is specified its the latest available. If a dependency is missing in mainline yocto then there is the joy of writing a slew of .bb and .bbappend files. The entire build ecosystem config is committed to VCS so that the toolchain and SDK can be rebuilt . But usually they're all running the latest available

[worthy cape]

This manifest is weird: https://steamdb.info/depot/238961/history/?changeid=M:5339855750552234382

Both the file listing and the downloaded tree is missing all the executable files.

Ah! They're moved to a separate depot https://steamdb.info/depot/238962/history/?changeid=M:78566953588493228, and there's a new macOS binaries depot at https://steamdb.info/depot/238963/history/?changeid=M:9191768501298523526

[worthy cape]

That's kind of interesting, as that means that we might have the same kind of bifurcation on macOS as on PC. Time will tell ^_^

[worthy cape]

[worthy cape]

Bleh, macOS is a silly platform.
I can use sysdir_start_search_path_enumeration to get hold of ~/Library/Application Support, but it has a literal ~ in the path by design.
I can use wordexp to tilde-expand but it also expands all sorts of shell syntax and splits the input into "words", so I get a two-element array out of that string.

glob will do, hopefully no-one will have wildcards in their username 😄

[worthy cape]

Bah, <filesystem> seems to be an XCode 11 thing, which supposedly means 10.15 minspec. Guess I won't be doing macOS tools any time soon.

[minor charm]

what version of osx are you running ?

[worthy cape]

10.13, last supported on this rock

[minor charm]

I see. Sorry but why can't you use $HOME ?

[worthy cape]

sysdir_start_search_path_enumeration gets me paths that explicitly use tildes. I don't want to reinvent the forms of tilde expansion.

[minor charm]

what is sysdir_start a part of ?

[worthy cape]

<sysdir.h>

[minor charm]

oh. I see

[worthy cape]

Directory paths returned in the user domain will contain "~" to refer to the user's directory.

I could try to sub it out myself and hope that they don't change that contract, but ended up just relying on glob to do it with GLOB_TILDE.

 $ clang++ -g -std=c++17 -o find_steam find_steam.cc&& ./find_steam 
/Users/hunter2/Library/Application Support/Steam/steamapps/common/Path of Exile

Works well enough, but without good compiler support in general I don't see much point in maintaining the platform.

[pseudo ocean]

hi, i'd like to extract assets (specifically audio) from Content.ggpk, is this currently possible/easy? i first tried this: https://github.com/jcmoyer/PoET which extracted some data from the ggpk, but i don't know what to do with many of the files after extracting them. i then tried https://github.com/OmegaK2/PyPoE which looked more promising, but it seemed quite broken on my machine and while i could at least get the ui to run, there were more errors when sorting the ggpk directory and the structure didn't show properly. i did find some stuff on github saying the ggpk format changed in harvest, could that be related to the issue? thanks for any advice

[worthy cape]

Hi!

There's indeed a difference in storage now since the tech patch:
Steam has a lot of assets loose in the installation directory (including .ogg voicelines), and the rest contained inside Bundle files (*.bundle.bin).
Standalone has a GGPK file like before, containing both those loose assets and the new Bundle files.

Any tool that used to handle the old GGPK files can get the loose files from a new GGPK with minor modifications, they just altered a version number in the GGPK chunk from 2 to 3.

For listing the contents of the bundles and extracting files from them, I have a tool.

If memory serves me right, the files you're likely to be interested in are all loose files, Ogg Vorbis in .ogg files and FMOD audio banks in .bank files.

As for PyPoE, it's still in flux as Omega is reworking it to handle the dual storage system.

[pseudo ocean]

my problem might already be solved then since i didn't let the first extractor fully run to see any .oggs yet. i'll try installing the steam version if that doesn't work since i didn't realise it had loose files. thanks for your help

[worthy cape]

You can sift through the listing of loose files and see which ones there are at https://steamdb.info/depot/238961/

[pseudo ocean]

lol i love how they named the files for each exile based on the attribute 'StrDexInt'

[worthy cape]

There's lots of such filename gems indeed 🙂

From an end-user perspective, there's also listings like https://oriath.net/audio-index.html

[pseudo ocean]

oh sweet, that's perfect

[worthy cape]

Heist isn't covered there yet, I forget if it was for spoiler reasons or something else.

[frank drift]

poedb also has a lot of audio

depending on what you're wanting

[rapid pagoda]

was initially for spoiler reasons, now just because I haven't gotten around to it

[worthy cape]

The best reason.

[simple ravine]

@frank kayak what is that terminal support thing you were playing around with called again?

[frank kayak]

@simple ravine terminal support thing?

[simple ravine]

yeah

[frank kayak]

that one

[simple ravine]

you were doing some funky things with the terminal

[frank kayak]

do you mean bitmap graphics?

[simple ravine]

among other things yeah

it reminded me of the good ol' dos days when you could change the resolution and do funky things

[frank kayak]

bitmap graphics is mostly sixel - vector graphics regis or tektronix 4014 or what its called

[velvet fog]

microtransaction search and price history https://poedb.tw/us/shop

[minor charm]

How are you retrieving that information for mtx ?

api/shop/microtransactions/specials ?

[velvet fog]

yap

[worthy cape]

What source are you querying for that? Seems to be missing the Demonic Spider Pet and Dancing Skeleton Pet.

The search boxes don't seem to do anything when invoked.

[velvet fog]

only discount history, not all microtransaction

not all microtransaction name match BaseItemTypes name, and it's hard to remapping

https://www.pathofexile.com/forum/view-thread/19091/page/103#p15721402

[tiny cargo]

Do you know if the api has the same limitation to 200 items as the special tab in the shop?

[velvet fog]

you can set limit to 1000 and it's worked, I got maximum 395 total before

https://www.pathofexile.com/api/shop/microtransactions/specials?type=active&limit=1000

[swift beacon]

I'm updating my 3D model viewer and it seems like GGG keeps making things more complex. Good for the game maybe but not for us data miners.

[worthy cape]

Hehe, running into something exciting?

All the ancillary data blobs in SMD are fun.

[swift beacon]

Today I found that now some of the .ao files inherit from other .ao files, so now I have to read an .ao file to get a reference to another .ao file to read that one.

[worthy cape]

Especially how they change order between v2 and v3.

I'm still not done with my .ao parser due to the irregularities and unknowns of the format.

[swift beacon]

Didn't used to be like that. They always had inheritance, but I never had to read more than one.

And now even the old models, like the PCs have been moved over to the newer format.

[worthy cape]

Either you trust the indentation and break on the broken files, or you have to have knowledge about the types of lines to know what to parse them as.

Upside of things, when assets are migrated between versions it can help with figuring the format out.

That's why I have all these old packs around in the first place.

[swift beacon]

The older ones I read read line by line as text and it worked fine. The newer ones are JSON, which is fine, but they aren't very consistent as to what properties they have.

That's true, comparing the new ones to the old ones might help.

I'm sure I'll figure it out, it's just annoying because generating each model is more work now.

[worthy cape]

ARM files are fun, they were up to v29 last I looked at them.

[swift beacon]

LOL nice

[worthy cape]

I long for the day where I have my new codebase at a point where I can work with file formats again.

[swift beacon]

At least it seems like a lot of the data I can ignore in order to get models that look pretty good. I'm not doing animations or fancy lighting or particle effects or anything like that.

[worthy cape]

Doodad lines have like four variants:

//     v14-ish:
//     60 15 6.19592 0 1 -7.8125 1 "Metadata/Terrain/Doodads/Forest/Gravestones/graveyard_fence_door_v01_01.ao" "Metadata/MiscellaneousObjects/Doodad"
//     v18:
//     20 80 -0.291014 0 0 -0.144994 0.989433 0 1 0 1 "Metadata/Terrain/Doodads/Beach/kolik_damaged_v01_01.ao" "Metadata/MiscellaneousObjects/Doodad"
//     v23: (same layout as v18, just for reference)
//     93 77 -0 0 0 0 1 0 1 -179.348 1.05 "Metadata/Terrain/Doodads/RuinedCity/Trees/OldTree_v01_03.ao" "Metadata/MiscellaneousObjects/Doodad"
//     v25:
//     218 233 2.94506 0 0 0.995176 0.0981067 0 0 1 -21.7391 1 "Metadata/Terrain/Doodads/Cave/treasure_pile_19.ao" "Metadata/MiscellaneousObjects/TreasurePile"

[swift beacon]

I've been ignoring the extra data at the end of the SMDs and so far the models still come out fine.

[worthy cape]

https://gitlab.com/zao/poe-rs/-/blob/master/src/formats/smd/types.rs

I've only mostly mapped out the data types, probably skeletons, lights, etc.

[worthy cape]

@swift beacon While you're around, have you considered exporting to and rendering glTF?

Even if you target a reasonable subset of it, it ought to be quite expressive.

Might not be able to capture all the VFX, but animations and richer materials ought to kind of work.

[swift beacon]

Haven't looked at it but sounds interesting.

[lethal nimbus]

is there any intent to support stash "folders" in the get-stash-items api endpoint?

[rapid pagoda]

Probably around the same priority as supporting the map stash tab, i.e. "when we get around to it, eventually"

[rancid lily]

How does Path of Building get character data? https://www.pathofexile.com/developer/docs/api-resources only mentions a few things. But I can get a character's items in JSON through e.g. https://www.pathofexile.com/character-window/get-items?accountName=ziddy5&character=markov_returns

Is there a more extensive documentation somewhere?

[hazy fiber]

its somewhere in here https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/master/Classes/ImportTab.lua

[grave wren]

https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/e0683a6c24d3e36bb1526f49dedfe75f97dff3e0/Classes/ImportTab.lua#L351 most likely

[hazy fiber]

thats just the list of characters not the data right?

[grave wren]

Ah shit true linked the wrong thing. You want Downloaditems and DownloadPassiveTree methods

[inland kestrel]

I'm sad. I'm able to put items in my public stash tabs, change zones, move them around, change zones, remove them.... and I'm noticing this activity not get reported by the public stash api

Looks like the public stash api is more for reporting items that have longer shelf life

[velvet fog]

the public stash api has 1 minute delay if you don't have oauth with it

[compact isle]

it's 5 minutes currently: https://www.pathofexile.com/forum/view-thread/2036957

folders and map stash support is targeted for the new stash API which relies on OAuth being released. The character window API will probably remain as is for backwards-compatibility until enough things have switched over

I forgot to mention how the change would affect the API ahead of time so apologies for that

[velvet fog]

where can I find the new API?

[compact isle]

it isn't available right now, OAuth needs to be properly released first

which is waiting on some terms of use stuff

[dense shore]

Any timeframe when we can expect it to roll out?

[lethal nimbus]

"when we get to it"

[candid dock]

does anyone know how trade search displays the data on the page? is there a script on the site that processes the fetched items and fills the html elements?

[candid dock]

i suppose there is no easy way to call that function with external data?

[civic crane]

window.app.$store, all vuejs app exposed in window.app

[hazy fog]

https://cdn.discordapp.com/attachments/609903318646652946/775219733959409684/image_from_ios.png

[worthy cape]

I'm sure you know this already but for channel exposition: it should work if you adapt the GGPK chunk's child_count field to be a version field instead, as the rest of the pack is identical. I don't think anyone has bothered doing that.
v3 is new PC standalone and otherwise identical, v4 is supposedly Mac standalone with UTF-32 strings.

[worthy cape]

I’ve got to panic learn enough of the language to understand some god-awful portal system for work, OpenOnDemand.

Not looking forward to it.

[golden bane]

There's some fun stuff like calling methods on integers directly:

5.upto(10) {|i| print i, " " }   #=> 5 6 7 8 9 10

Now that I look at it, the block syntax reminds me of Rust a bit, maybe that's where Rust got it from

[digital parrot]

Hello there. Where can I ask about possible bugs or problems calculating DPS in Path of Building ?

[golden bane]

@digital parrot Open an issue on GitHub if you believe there is an error in the calculations

[digital parrot]

Ok.

GitHub wants me to register. Is there any other way?

Can someone with a GitHub account forward this for me?

[golden bane]

No

[digital parrot]

Guess I leave it be then ^^

[swift beacon]

Well, I think I figured out the latest version of the .mat files (at least as much as I need to for my purposes). I'm generating the 3D models for all of the characters, NPCs, monsters, chests, items, etc. without trouble now, regardless of the .mat file version and format.

I found a couple of new monsters/bosses that I hadn't seen before and which don't seem to be released yet. Links in case anyone is curious (spoilers, obviously): https://i.imgur.com/HcQKllu.png https://i.imgur.com/rDQqBHi.png

[worthy cape]

Do you have any docs on it?

[swift beacon]

No, but one of these days I might write something up.

A lot of the stuff I still don't understand, but it seems like stuff I don't need to unless I want to start trying to add in better lighting effects etc.

[worthy cape]

I wish I had the peace of mind and time to get back into formats again, work is a hellscape atm.

Let's take a C++ programmer and build engineer and put them on modifying a web application in Ruby. That's a good use of resources and definitely not stressful.

</rant>

[swift beacon]

Yeah, I've seen management make some awfully silly decisions. I think non-programmers think any computer person = any computer person.

[worthy cape]

This is worse, this is arguably technical people, which makes it worse.

I'm good at the things I do, but the job description includes a lot of ops stuff I have no interest nor competence in.

[swift beacon]

Intelligent allocation of resources is definitely a skill many don't have.

[frank kayak]

offtopic: EU wants a generic key in every application ♥️
internal EU paper https://files.orf.at/vietnam2/files/fm4/202045/783284_fh_st12143-re01en20_783284.pdf
article in German: https://www.ccc.de/de/updates/2020/cryptowars-2020
and one in English: https://apnews.com/article/technology-data-privacy-europe-fdf47545b487f545ba9f48e38d379a94

I hope the discontent about this spreads a bit...

[golden bane]

@frank kayak That's not quite true and also no reflected in the contents of the proposal draft. The last time members of the European Council advocated for access to E2E encryption, they wanted a master key. Having failed that, the current proposal doesn't contain any implementation details at all, passing the responsibility to the third parties providing the encryption instead. So while putting the proposal into effect would be the end to secure encryption, now it's also possible in exciting new ways like user/chat/whatever-specific master keys, MITM attacks or "Quellen-TKÜ" (no idea how to translate this into English properly). Arguably the new proposal is potentially even worse.

[frank kayak]

Quellen-TKÜ would be wiretapping at the source i guess?

to be correct i skimmed through most...

but the endangering of secure encryption is what bugs me

[golden bane]

Something like "install malware to the target device to send (or retain and later send) data before encryption/after decryption"

[frank kayak]

yeah

[frank drift]

good to see the US doesn't have a monopoly on that kind of stupidity

[simple ravine]

@frank kayak lol wtf, that's like a 180 pivot from the whole privacy bs they've been touting

wtf

Vienna attack: 2nd of November.
...

Delegations will find in attachment the revised version1 of the Draft Council Resolution on
Encryption. It reflects the comments received from the Member States before and during the
informal VTC meeting of JHA Counsellors (Encryption) on __3 November 2020.
__

They were just waiting for the right moment, I guess.

I wonder how they'll be able to enforce this, though...

[tiny cargo]

Is it really a turnaround? Private company collecting data = bad, government collecting data = good is their view.
The sad thing is that Slovak Department of the Interior informed the Austrians via Europol that he tried to buy ammunition. The Austrian services failed to observe or to arrest him, even Karl Nehammer admitted that they made mistakes. So with enough information they failed and now they want more information to do what?
I doubt that it will be enforced. Some court will rule against it, sounds highly unconstitutional to me.

[frank drift]

during the hearings in the US about the encryption backdoor bill I recall a congressperson asking a pentagon official what specific attack it would've prevented, the official could not name any

[rapid pagoda]

@fickle yew any thoughts on removing non-red beasts from beasts? there's some crazy price spikes showing up rn for random yellow beasts

[grave wren]

Kinda wondering if releasing Poe for Mac was a bad idea with arm models coming now. Do we know whether apples translation app results tend to do good?

[minor charm]

What kinda troubles do you see ARM chips introducing for PoE ?

[fickle yew]

@rapid pagoda might make sense yea

[grave wren]

Don't think Poe runs on arm without the translation layer so I was wondering if anything is known about performance loss for x86_64 software

[frank drift]

maybe you have to play PoE Mobile on Arm Macs

[hazy fiber]

depends on what instructions are used and what type of arm chip

[worthy cape]

I’d expect the transition story to be reasonable, from what I’ve heard from a browser vendor it’s not that bad to port to and there’s been dev machines out.

It’s been a while since I had to deal with fat binaries, I wonder if they make a return 🙂

[frank kayak]

https://icculus.org/fatelf/

its not supported on regular linux distros though :/

Riot Games uses Go https://redd.it/jsdulp

[pseudo ocean]

well it must be good then

[minor charm]

What is the translation layer in this context ? also was there any mention of the specific arm arch they were planning on using ?

[rapid pagoda]

@minor charm The "translation layer" is the thing deep in the OS which implements a x86 emulator/JIT. Apple uses the brand name "Rosetta 2" to refer to this particular implementation.

Apple's current SoCs are using some variant of ARMv8-A; not clear exactly which subrevision is implemented. It's a custom implementation, so it won't line up with any of ARM's standard Cortex-A(whatever) cores

[minor charm]

ah ok, a bit like qemu

[rapid pagoda]

specifically like qemu-user-binfmt, if you've ever used that configuration 🙂

[minor charm]

I see. I don't think I used it. I think it was qemu-arm-static for Cortex-M series devices

[rapid pagoda]

oh, yeah, qemu-system is a different thing entirely 🙂 anyways, the end result with qemu-user-binfmt is that you can run non-native executables as if they were native executables, transparently; Rosetta has the same effect

[rotund oxide]

Does anyone know how to get more data using the API I seem to be only getting about 600 stashes whenever I use a get request.

[rapid pagoda]

That's normal. Follow the next_change_id references for more

[worthy cape]

I'm having the worst time with the Mac client. My own MBP is too old for Catalina so can't even launch the patcher.
VirtualBox doesn't expose even AVX, let alone AVX2.
qemu with KVM has the ability to remove feature flags from CPUID but still runs instructions you told it to remove.
qemu with TCG emulation doesn't support AVX yet.

As a bonus, you need to turn off SIP protection to be able to attach a debugger at all to a notarized application.

Right now I'm trying to run qemu with KVM on my actual Mac under Fedora, so that I can try running a newer macOS on the hardware that was considered too legacy to upgrade to it.

[frigid nova]

https://twitter.com/gvanrossum/status/1326932991566700549
(creator of python)

Guido van Rossum (@gvanrossum)

I decided that retirement was boring and have joined the Developer Division at Microsoft. To do what? Too many options to say! But it’ll make using Python better for sure (and not just on Windows :-). There’s lots of open source here. Watch this space.

Retweets

1814

Likes

12012

Twitter · Nov 12, 2020, 5:00 PM

[worthy cape]

Heh, it actually worked...

macOS 10.15 via qemu KVM on Linux on a MacBook Pro that's too old for anything newer than macOS 10.13.

[frank kayak]

how slow?

[worthy cape]

As the local disk is too small, the VM runs over NFS.

[frank kayak]

eh fast...

[worthy cape]

Surprisingly interactive.

[frank kayak]

oh

[worthy cape]

Takes a while to boot, but it can run Discord in Safari and a debugger.

[frank kayak]

i tried macos on kvm a few months ago - but got glitched graphics after booting. I think it doesnt like my cpu

[worthy cape]

Biggest struggle was figuring out how to change the CSR flags in the Clover firmware so that I could attach a debugger.

There's a bit of glitching in the terminal tabs.

[frank kayak]

csr?

[worthy cape]

It's all software rendering too, of course, as I don't have any passthrough.

Not sure what CSR stands for, but you use csrutil to modify System Integrity Protection.

The Apple infrastructure to prevent unsigned kernel extensions, prevent DTrace and debugger attaching to hardened apps, etc.

[frank kayak]

sounds like a bit of work...

[worthy cape]

Took a day or so of curiosity.

First trying to disassemble on real hardware, then VirtualBox, then qemu+KVM on a good machine, then qemu+TCR, then qemu+KVM on real hardware.

It's always nice to have a deep rabbit hole to go down and learn things in.

[frank kayak]

agree

i tend to choose problems that dont offer much when solved

i prob try macos on kvm again when i can create my new computer. i will question you then

[worthy cape]

Upside of things, I've got a reasonably well-working macOS installation now too on the i7-6700K.

[frank kayak]

nice

i have it bookmarked so it was prob that one i tried

[worthy cape]

I ran this: https://github.com/foxlet/macOS-Simple-KVM

(sorry for wrong link before, so many tabs open with slivers of information)

[frank kayak]

prob around 1k firefox tabs again myself 🙂

are you using tab groups?

[worthy cape]

Nah, just Tree Style Tabs (occasionally) and a few windows.

[frank kayak]

ah did use that at some point

that reminds me of my stash tabs. somehow i have only around 880 tabs now and im pretty sure i did hit 1k tabs or was far above 950

and shouldnt harvest stuff (hortis + seeds) already be removed? i still have them. not sure if thats how it should be.

does anyone else still have hortis + seeds?

its this one: https://addons.mozilla.org/de/firefox/addon/panorama-tab-groups/

was damn happy about the folders. all now sorted in folders - most is in the unsorted folder... - white siege axe tabs in the chance folder

[worthy cape]

I used to have some addon that slurped all the tabs down into groups or lists to restore later, but I didn't like how it lost me state.

[frank kayak]

@worthy cape why do you use Clover? https://sourceforge.net/projects/cloverefiboot/ this one?

[worthy cape]

It's what came stock on the Simple KVM stuff above.

[frank kayak]

ok

[worthy cape]

I guess it helps with getting all those pesky settings right on less-than-genuine hardware.

[frank kayak]

dont want to use my brain today much anymore prob playing a bit

[worthy cape]

Worst of all is that I've done most of this on the PoE machine, so I haven't played at all 😄