Integrate Database and Real-Time Data

[mellow helm]

I have a task with 2 database sql server and mongoDB:
MongoDB have 1 table with name Employee
SQL Server have 1 table with name Employment with 1 field is employee_id

My mission is:

1. Combine 2 database from 2 difference DBMS and create dashboard.
2. I need real-time data for dashboard (it mean when i insert new record in database, dashboard will change)

My solution is using web-socket but guide i find is just real-time chat-app
Do anyone have example suitable for my case? Or any solution for 2 mission above. Help

P/S: First time i create dashboard and just leaning Golang 1 month. It's so confused TT

[split knoll]

I'm sure you can use the ws chat example to implement whatever you need. How are you polling the database for "real time" updates?

[raw hill]

I don't think you need websockets for this kinda of information

[mellow helm]

I just code follow chat-app to create but i need call from db while i just found some dummy demo about chat-app , so i jsut cop, paste, gpt, everything.
Sorry to do this. I know do this is wrong but deadline TT

can u share steps to approach this to newbie like me

[raw hill]

what are you using for your UI ?

[mellow helm]

apex chart : js , and some html, css template

[raw hill]

how often really would be the user be changing the data? because this kind of information i dont see he updating it all the time to justify using websockets.

you could for example, when making a fetch call that will make any kinda of modification, have the related data make a refetch call

[mellow helm]

just when i use post api to create new employee or other record, the data in dashboard is change (not F5)

[raw hill]

you could implement your own refetcher logic, inspired by the this idea.
https://docs.solidjs.com/reference/basic-reactivity/create-resource
but only the refetch method

for example, when you create a new employee or update a existing one, you make a new http call for whatever you're displaying. If your displaying a single employee details, just refetch him and update the ui, the same if you're displaying a list of them.

[mellow helm]

yah i still think this in frontend

but my case is need handle it in BE

its require

[raw hill]

its rendered by the server?

[mellow helm]

yes sir, i need it

[raw hill]

what are you serving your end user? html ?

[mellow helm]

yes sir, i don't know why this is require, i think we need send signal to frontend and make it upload when change but i dont know it can be call is refresh or not

hic, i so confused

but sir, this just my second require

i need solution for integration 2 db form 2 differnt dbms how i can get that

the second require not to strictly if i not done, but first require , i not done: im dead

[raw hill]

so far, how are you feeding your chartjs data, or you didnt begin it yet?

[mellow helm]

i just using axios to call api, build count, select in get api and render it in frontend

using existing template

Im sorry for to stupid, but anything i did in coding just CRUD api in 1 database

So i don't know how to combine it in 2 dbms

[raw hill]

before serving your data, create some kind of layer to deal with fetching both databases and normalizing the data for your needs (axios call).

[mellow helm]

what layer for it sir, can i have some example like this, 1 Employement(have employee_id,..) in 1 dbms and employee(id, ...) in another dbms.

If 1 dbms i just join and create foreign key, but 2. Im so confused.

Some friend talk me need create middleware but i not know what is middleware and what case of middleware for this case hic TT

[raw hill]

type Employee struct{}

type Postgres struct {
    // your postgres or whatever database you're using access ex: *sql.DB
}

func (pg *Postgres) GetEmployeesById(id any) ([]Employee, error) {
    return nil, nil
}

type MySql struct {
    // your MySql or whatever database you're using access ex: *sql.DB
}

func ListEmployees(pg *Postgres, mysql *MySql, id any) ([]Employee, error) {
    employeesFromPostgres, err := pg.GetEmployeesById(id)
    if err != nil {
        return nil, nil
    }

    employeesFromMySql, err := mysql.GetEmployeesById(id)
    if err != nil {
        return nil, nil
    }

    // your logic to normalize both database
    normalizedData, err := NormalizeData(employeesFromPostgres, employeesFromMySql)
    if err != nil {
        return nil, nil
    }

    return normalizedData, nil
}

[mellow helm]

Thanks sir, i will try to code base on this. I think it make me find out solution! Thanks u so much