well this attempt was fun, still learning how this filtering works. it gets rid of the whistling for the most part, but the dropoff is so steep that now SMS sounds kinda bad... however, how did SMS sound on a real genesis with a power base converter anyways? was it muffled on the high end or was it ear bleedingly high pitched like the original sms console? i don't know the answer to that. i brought back the 16khz iir filter sorg had at the original commit and then switched the cegen's outclk to use the same clock as the z80 (which is 1/15'th the speed of the master clock which is used as the input). it made the after burner II whistling sound very hard to hear, so it's an improvement. there is now a little imbalance, but that can be adjusted later. i gotta go to bed, didn't realize what time it was lol.
anyways I do think that the filter should use the clock enable that is based on what the clock input speed was for the real genesis. i looked at the filtering in the tgfx16 core and the psg filter is done this way, it's using a ce from a divider of the master clock.