Console Logging Encoding Problems | SpigotMC | Page 1

dark quest Jun 2, 2022, 5:34 PM

#

package me.lianecx.smpplugin;

import java.util.ArrayList;
import java.util.List;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.appender.AbstractAppender;

public class ConsoleLogger extends AbstractAppender {
    private boolean isLogging;
    private List<String> loggedData;

    public ConsoleLogger() {
        super("SMP-Plugin", null, null, true);

        isLogging = false;
        loggedData = new ArrayList<>();
    }

    public void startLogging() {
        isLogging = true;
    }

    public void stopLogging() {
        isLogging = false;
    }

    public void clearData() {
        loggedData.clear();
    }

    public List<String> getData() {
        return new ArrayList<>(loggedData);
    }

    public void log(String message) {
        if (isLogging) {
            loggedData.add(message);
        }
    }

    @Override
    public boolean isStarted() {
        return true;
    }

    @Override
    public void append(LogEvent event) {
        log(event.getMessage().getFormattedMessage());
    }
}

solar salmon Jun 2, 2022, 5:34 PM

#

can you throw that in a paste?

#

?paste

dark quest Jun 2, 2022, 5:34 PM

#

probably doesnt work in threads

#

@tawny bison

#

?paste

solar salmon Jun 2, 2022, 5:35 PM

#

https://paste.md-5.net/

dark quest Jun 2, 2022, 5:35 PM

#

https://pastebin.com/fdxEyKCZ

#

https://paste.md-5.net/wobifejake.java

#

I also tried this as the constructor ```java
super("SMP-Plugin", null, PatternLayout.newBuilder().withCharset(StandardCharsets.UTF_8).build(), true);

#

Didnt really change anything

#

solar salmon Jun 2, 2022, 5:52 PM

#

Test if its teh formatting that is breaking it. In append use event.getMessage().toString() to see what the base string is

dark quest Jun 2, 2022, 5:57 PM

#

And again, still logging correctly

solar salmon Jun 2, 2022, 5:58 PM

#

ok, lets try and fix a few things. You shoudl move off a deprecated constructor java super("SMP-Plugin", null, PatternLayout.newBuilder().withCharset(StandardCharsets.UTF_8).build(), true, Property.EMPTY_ARRAY);

dark quest Jun 2, 2022, 5:59 PM

#

note that im using api version 1.13 and java 8

dark quest Jun 2, 2022, 5:59 PM

#

solar salmon ok, lets try and fix a few things. You shoudl move off a deprecated constructor ...

what is Property

#

there are a lot of imports avaIlable

solar salmon Jun 2, 2022, 6:00 PM

#

no clue

dark quest Jun 2, 2022, 6:00 PM

#

this should be the correct one

#

but theres no EMPTY ARRAY property on it

dark quest Jun 2, 2022, 6:00 PM

#

solar salmon ok, lets try and fix a few things. You shoudl move off a deprecated constructor ...

where did you find that

#

and how do you know its deprecated

solar salmon Jun 2, 2022, 6:00 PM

#

the constructor you are using says deprecated

#

atleast in my IDE for 1.18.2

dark quest Jun 2, 2022, 6:01 PM

#

theres only these 2 overwrites

dark quest Jun 2, 2022, 6:01 PM

#

solar salmon atleast in my IDE for 1.18.2

okay then thats the problem probably

dark quest Jun 2, 2022, 6:01 PM

#

dark quest note that im using api version 1.13 and java 8

^

solar salmon Jun 2, 2022, 6:02 PM

#

does your 1.13 have the log4j exploit fix?

#

it was fixed in Spigot but probably not in paper

dark quest Jun 2, 2022, 6:02 PM

#

what was fixed?

#

Im not using the paper api

#

im using a paper server though

solar salmon Jun 2, 2022, 6:06 PM

#

it was a remote code execution exploit in log4j

#

there is a startup switch to preven tit... one sec

#

-Dlog4j2.formatMsgNoLookups=true

dark quest Jun 2, 2022, 6:08 PM

#

that was definitely fixed in paper as well

#

But youre going offtopic arent you?

solar salmon Jun 2, 2022, 6:08 PM

#

yep

#

ok lets test, in your appender try java String msg = String.format(Locale.ENGLISH, event.getMessage().getFormattedMessage(), (Object) getPropertyArray()); log(msg);

#

That way we can see of the encoding is wrong going in

#

see what msg is after formatting

dark quest Jun 2, 2022, 6:11 PM

#

getPropertyArray()?

solar salmon Jun 2, 2022, 6:12 PM

#

just fetches the array that the appended was setup with

dark quest Jun 2, 2022, 6:12 PM

#

well i dont have that method

#

where are you getting it from?

solar salmon Jun 2, 2022, 6:13 PM

#

does it not exist for yiou?

dark quest Jun 2, 2022, 6:13 PM

#

nope

solar salmon Jun 2, 2022, 6:13 PM

#

then use Property.EMPTY_ARRAY

dark quest Jun 2, 2022, 6:13 PM

#

i dont have that either

solar salmon Jun 2, 2022, 6:13 PM

#

oh

#

it should be a Property[]

dark quest Jun 2, 2022, 6:13 PM

#

i can just leave it out

solar salmon Jun 2, 2022, 6:13 PM

#

k

dark quest Jun 2, 2022, 6:13 PM

#

if its empty

#

well it still shows the same thing

#

Btw, maybe we can change the whole fundamental idea

#

What im ultimately trying to do is just receive the response of a command which im executing through the plugin

#

This is unfortunately very hard to do and ive tried some things but the best working one was using the logger which starts logging before i execute the command and immediately stops after

#

Maybe theres an easier way of doing this

#

It would solve this problem and be overall probably cleaner

solar salmon Jun 2, 2022, 6:20 PM

#

I can't see that log4j would be trying to alter teh encoding at all. There has to somethign funky with how the string is getting to the logger

dark quest Jun 2, 2022, 6:22 PM

#

hmm

dark quest Jun 2, 2022, 6:22 PM

#

dark quest What im ultimately trying to do is just receive the response of a command which ...

but you dont have an idea for another method of doing this?

solar salmon Jun 2, 2022, 6:23 PM

#

Not really. The command system isn;t designed to get anything more informative than a yes/no

#

I'll assume you are sending commands to other plugins

dark quest Jun 2, 2022, 6:24 PM

#

?

#

wdym

solar salmon Jun 2, 2022, 6:24 PM

#

via dispatchCommand?

dark quest Jun 2, 2022, 6:24 PM

#

yea

solar salmon Jun 2, 2022, 6:25 PM

#

is there a reason you can;t use the plugins api?

dark quest Jun 2, 2022, 6:25 PM

#

i read theres a way of overriding the sendMessage method of a command sender but that didnt really work for me

dark quest Jun 2, 2022, 6:25 PM

#

solar salmon is there a reason you can;t use the plugins api?

oh im not executing commands of other plugins

#

im executing vanilla commands

solar salmon Jun 2, 2022, 6:26 PM

#

ah

#

relying on the logger for that is going to be unreliable as it starts out async

dark quest Jun 2, 2022, 6:27 PM

#

dark quest i read theres a way of overriding the sendMessage method of a command sender but...

ik but theres not really another way of obtaining the response other than this

solar salmon Jun 2, 2022, 6:27 PM

#

probably not

#

where are you displaying the messages that you see the wrong encoding? If as you say they are correct in console and log

dark quest Jun 2, 2022, 6:48 PM

#

in my second app

#

in the console of my second app

#

where im sending the messages to

solar salmon Jun 2, 2022, 6:50 PM

#

in which case it seems the method you are using to send the string across is wrong

#

or that console is not UTF-8

dark quest Jun 2, 2022, 6:55 PM

#

oof

#

maybe

#

How could i find out

#

its a js app and i tried this

    console.log(Buffer.from(respMessage, 'utf-8').toString())

#

which shouild just convert it to utf8

solar salmon Jun 2, 2022, 7:05 PM

#

I don;t do js so not clue

#

its been years for me since I touched js

#

Node

dark quest Jun 2, 2022, 7:06 PM

#

do you have any pseudo-code?

solar salmon Jun 2, 2022, 7:06 PM

#

how are you sending to js?

dark quest Jun 2, 2022, 7:06 PM

#

http response

solar salmon Jun 2, 2022, 7:07 PM

#

can you read teh encoding on the response?

#

it should be in teh header info

dark quest Jun 2, 2022, 7:07 PM

#

probably

#

its not really in there

#

i can specify the header though

solar salmon Jun 2, 2022, 7:16 PM

#

you can also before sending ensure it is UTF-8 by string.getBytes(StandardCharsets.UTF_8)

#

include the encoding in teh header and Node should handle it

dark quest Jun 2, 2022, 7:28 PM

#

okay i need a break 😅

#

Ill come back tomorrow

#

its not setting the header for some reason

solar salmon Jun 2, 2022, 7:29 PM

#

ok

dark quest Jun 2, 2022, 7:40 PM

#

solar salmon ok

Btw i almost forgot. Another hint that the logger is outputting wrongly encoded strings is that the stripColor method couldnt remove the comor coded and we had to make our own one with the special regex

solar salmon Jun 2, 2022, 7:40 PM

#

thats true

dark quest Jun 2, 2022, 7:40 PM

#

And the regex is checking for \x7f

solar salmon Jun 2, 2022, 7:41 PM

#

instead of hooking into log4j perhaps you shoudl hook the console

#

This would give you what is actually being pushed to the console

#

https://github.com/ElgarL/GroupManager/blob/master/src/org/anjocaido/groupmanager/utils/GMLoggerHandler.java

dark quest Jun 2, 2022, 7:55 PM

#

Id have to remove the prefixed though

#

Like the time and the sender

dark quest Jun 2, 2022, 9:11 PM

#

Btw, i just wanted to thank you for your help. This really means a lot :D

dark quest Jun 3, 2022, 11:02 AM

#

solar salmon <https://github.com/ElgarL/GroupManager/blob/master/src/org/anjocaido/groupmanag...

At which logging levels are commands sent?

solar salmon Jun 3, 2022, 11:02 AM

#

info

dark quest Jun 3, 2022, 11:02 AM

#

all responses?

#

the error ones too?

solar salmon Jun 3, 2022, 11:03 AM

#

question. the string from yesterday with the 7F at the front. was that the string whilst IN the logger, or after you sent it across to your app?

dark quest Jun 3, 2022, 11:03 AM

#

The \x7f was in the second app

#

Because log4j displayed it correctly

solar salmon Jun 3, 2022, 11:03 AM

#

in that case, there is nothing wrong with your logger

dark quest Jun 3, 2022, 11:03 AM

#

in the minecraft console

dark quest Jun 3, 2022, 11:04 AM

#

dark quest Btw i almost forgot. Another hint that the logger is outputting wrongly encoded ...

This

solar salmon Jun 3, 2022, 11:04 AM

#

the issue is purely in how you are sending it to the app

dark quest Jun 3, 2022, 11:04 AM

#

it isnt

#

I think

#

i do not think its the transfer

solar salmon Jun 3, 2022, 11:04 AM

#

I do.

dark quest Jun 3, 2022, 11:04 AM

#

dark quest Btw i almost forgot. Another hint that the logger is outputting wrongly encoded ...

Because of this

#

Then why does this regex work and the stripColor one doesnt?

\x7F([(\da-fA-F)])

solar salmon Jun 3, 2022, 11:05 AM

#

is that regex done in your MC plugin code or on your app?

dark quest Jun 3, 2022, 11:05 AM

#

#

thats the bukkit regex

dark quest Jun 3, 2022, 11:06 AM

#

solar salmon is that regex done in your MC plugin code or on your app?

its done in my mc plugin

#

\u00A7

#

it uses the unicode char

solar salmon Jun 3, 2022, 11:06 AM

#

ok

dark quest Jun 3, 2022, 11:06 AM

#

dark quest Then why does this regex work and the stripColor one doesnt? ```regex \x7F([(\da...

and we use a different char

#

and ours work

#

[0-9A-FK-OR]
btw this is correct

#

we didnt include the reset (r), underlined (l), bold etc

solar salmon Jun 3, 2022, 11:07 AM

#

K-OR?

#

ah

#

we shouldn;t have to

dark quest Jun 3, 2022, 11:07 AM

#

?

solar salmon Jun 3, 2022, 11:07 AM

#

ah no ignore me

#

I was just thinking of the 16

dark quest Jun 3, 2022, 11:08 AM

#

ohh

#

and it does (?i) for ignoring case?

#

yea

solar salmon Jun 3, 2022, 11:09 AM

#

yes it needs /i

dark quest Jun 3, 2022, 11:09 AM

#

(?i)\x7F[\dA-FK-OR]

#

this is our final regex

solar salmon Jun 3, 2022, 11:09 AM

#

did you try with the console logger to see what you get there?

dark quest Jun 3, 2022, 11:10 AM

#

im trying rn

solar salmon Jun 3, 2022, 11:10 AM

#

k

dark quest Jun 3, 2022, 11:10 AM

#

thats why i wanted to know the level

#

is it always INFO?

#

also for syntax exceptions

#

like when it says

#

An entity is required blabla

#

or

solar salmon Jun 3, 2022, 11:10 AM

#

I'd not be suprised if it was a log4j issue. That thing is a bitch to work with

dark quest Jun 3, 2022, 11:10 AM

#

awdawd....<------HERE

#

ill just try always info

solar salmon Jun 3, 2022, 11:11 AM

#

you could use .ALL

#

pretty sure there is an ALL level

#

well for now you can ignore the level. we just need the message

#

plugin#getLogger().setUseParentHandler(false);
Plugin#getLogger().addHandler(new LogHandler());```

dark quest Jun 3, 2022, 11:43 AM

#

solar salmon ```java plugin#getLogger().setUseParentHandler(false); Plugin#getLogger().addHan...

why do we need this?

solar salmon Jun 3, 2022, 11:44 AM

#

that attaches the LogHnadler to teh logger

dark quest Jun 3, 2022, 11:44 AM

#

so its not a global logger?

solar salmon Jun 3, 2022, 11:44 AM

#

you add it to the plugins logger

#

or bukkit logger

dark quest Jun 3, 2022, 11:44 AM

#

but that logger doesnt actually log

dark quest Jun 3, 2022, 11:44 AM

#

solar salmon or bukkit logger

oh

#

well um

#

it threw a bunch of errors

#

stackoverflow

#

public class ConsoleLogger extends ConsoleHandler {
    private boolean isLogging;
    private final List<String> loggedData;

    public ConsoleLogger() {
        isLogging = false;
        loggedData = new ArrayList<>();
    }

    public void startLogging() {
        isLogging = true;
    }

    public void stopLogging() {
        isLogging = false;
    }

    public void clearData() {
        loggedData.clear();
    }

    public List<String> getData() {
        return new ArrayList<>(loggedData);
    }

    public void log(String message) {
        if (isLogging) {
            loggedData.add(message);
        }
    }
    @Override
    public void publish(LogRecord record) {
        System.out.println(record.getLevel().getName());
/*        if (record.getLevel() == Level.INFO) {
            log(record.getMessage());
        }*/
        log(record.getMessage());
    }
}

#

i think i cant sysout in the publish method

solar salmon Jun 3, 2022, 11:46 AM

#

you can

dark quest Jun 3, 2022, 11:46 AM

#

]

#

well its at that line

#

at the sysout line

#

ohh

solar salmon Jun 3, 2022, 11:47 AM

#

ah paper, it hates sysout

dark quest Jun 3, 2022, 11:47 AM

#

yea

#

thats what i said

#

ill download the new version real quickj

#

and i also removed the sysout

#

no errors now

dark quest Jun 3, 2022, 12:07 PM

#

It doesnt seem to work

solar salmon Jun 3, 2022, 12:07 PM

#

gets nothing?

dark quest Jun 3, 2022, 12:07 PM

#

yeah

#

data is emptz

solar salmon Jun 3, 2022, 12:08 PM

#

odd. I know it works at teh plugin level. I've not used it on the Bukkit logger

#

but it should. identical loggers

#

try leaving out the setParentHandlers

#

allow all handlers to parse the messages

dark quest Jun 3, 2022, 12:21 PM

#

it looks like its just the plugin logger

solar salmon Jun 3, 2022, 12:21 PM

#

ah thats a bummer then

dark quest Jun 3, 2022, 12:23 PM

#

#

\

#

this maybe?

#

add the handler to this one

solar salmon Jun 3, 2022, 12:24 PM

#

or use getLogger passing the Main class of Spigot/paper

dark quest Jun 3, 2022, 12:24 PM

#

?

#

wdym

solar salmon Jun 3, 2022, 12:24 PM

#

From Spigot Main class Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);

#

logger being java.util.logging.Logger;

dark quest Jun 3, 2022, 12:25 PM

#

whats main

#

Main.class

solar salmon Jun 3, 2022, 12:25 PM

#

yes, Spigots Main.class

dark quest Jun 3, 2022, 12:26 PM

#

i dont have Main

solar salmon Jun 3, 2022, 12:26 PM

#

its called Main as its an executable

dark quest Jun 3, 2022, 12:26 PM

#

where does it come from

solar salmon Jun 3, 2022, 12:26 PM

#

ah you'd need NMS

dark quest Jun 3, 2022, 12:26 PM

#

?

solar salmon Jun 3, 2022, 12:26 PM

#

org.bukkit.craftbukkit.Main

dark quest Jun 3, 2022, 12:26 PM

#

solar salmon ah you'd need NMS

ah okay

#

this doesnt work?

solar salmon Jun 3, 2022, 12:27 PM

#

nope

dark quest Jun 3, 2022, 12:28 PM

#

hmm

#

getServer.getLogger also didnt work

solar salmon Jun 3, 2022, 12:29 PM

#

how about Logger.getLogger("Minecraft");

dark quest Jun 3, 2022, 12:30 PM

#

how do you know the name is Minecaft?

solar salmon Jun 3, 2022, 12:31 PM

#

its in stash

#

https://hub.spigotmc.org/stash/projects/SPIGOT/repos/craftbukkit/browse/src/main/java/org/bukkit/craftbukkit/CraftServer.java#252

dark quest Jun 3, 2022, 12:31 PM

#

?

#

ah

#

still nothing

#

setUseHandlers(false)?

solar salmon Jun 3, 2022, 12:32 PM

#

leave that line out

dark quest Jun 3, 2022, 12:32 PM

#

i did

solar salmon Jun 3, 2022, 12:32 PM

#

thats for overriding current handlers to only use ours

dark quest Jun 3, 2022, 12:32 PM

#

should i enabel it?

solar salmon Jun 3, 2022, 12:32 PM

#

default is enabled

dark quest Jun 3, 2022, 12:33 PM

#

i mean disable

solar salmon Jun 3, 2022, 12:33 PM

#

nope

dark quest Jun 3, 2022, 12:33 PM

#

okay

#

then it doesnt work

solar salmon Jun 3, 2022, 12:33 PM

#

probably not 🙂

dark quest Jun 3, 2022, 12:33 PM

#

I didnt disable it

#

And it didnt work

solar salmon Jun 3, 2022, 12:33 PM

#

I'm at a loss then

dark quest Jun 3, 2022, 12:34 PM

#

So this is the end result...

#

ig i dont have any other idea too

#

ill just use the old logger

#

the log4j appender

solar salmon Jun 3, 2022, 12:36 PM

#

I guess you have no other choice

dark quest Jun 3, 2022, 12:36 PM

#

I mean we did manage to fix the color codes

#

its just the other unicode chars

solar salmon Jun 3, 2022, 12:37 PM

#

other unicodes are unchanged so they have to be transmitting incorrectly

#

somethign between you storing the string and getting it to your app

dark quest Jun 3, 2022, 12:48 PM

#

what?

#

This wouldnt make sense

#

if the § is encoded weirdly then so are all the other non-english chars

#

its not a transmission problem

dark quest Jun 3, 2022, 12:50 PM

#

solar salmon somethign between you storing the string and getting it to your app

dont you think so too?

solar salmon Jun 3, 2022, 12:52 PM

#

I've no idea how its messing up the color code, but I don;t think its unicode related

#

if its not related to the encoding then the other characters should be fine

#

if it is the encoding then ALL unicodes will be broken

dark quest Jun 3, 2022, 1:15 PM

#

solar salmon if it is the encoding then ALL unicodes will be broken

and thats whats happening

#

Anyways, thank you for your amazing help and effort :D

#

Ive reached my main goal which is removing the color codes

#

uwu

#Console Logging Encoding Problems