Lightbug HTTP: Mojo web framework

[radiant verge]

https://github.com/saviorand/lightbug_http

[radiant verge]

by @inland scaffold

[lost gorge]

hi, I have been experimenting with Lightbug as a serving interface for https://github.com/alainrollejr/mocodes

A couple of questions which I don't know are worth filing change requests for. First one is I actually needed HttpService's func to be allowed to change self (essentially to keep state over incoming requests). For now solved that through pointers but would be easier if I could just change some self.counter value from func.

Second one, my test client sends POST messages of a certain length L (smaller than 4096 bytes). Is there a way for the serving handler only to be called when the full L bytes have been received on the socket ? Mostly that is the case but random occurences happen whereby func() is called on body with unexpected length way smaller than L (requiring me to keep even more local state).

Last, my application actually does numerical work on Int8 bytes. So I think if there would be a way that Bytes() could actually be a DTypePointer into Int8 rather than a List of Int8, might be beneficial for my application even though List has this scary "steal_data" functionality

[radiant verge]

@inland scaffold fyi ^

[inland scaffold]

@radiant verge Thanks a lot for tagging!
@lost gorge great questions, I think these are definitely worth making issues for. I'll create some issues today and link here. Then let's continue the discussion on GitHub if that works 👍

[hazy terrace]

@inland scaffold you may both be interested in the Bytes implantation in Basalt, in the utils folder

[inland scaffold]

Ah perfect, thanks a lot for the rec!!

@lost gorge made these three issues, tried to rephrase them based on my understanding. Let me know if I made mistakes somewhere

1. HTTPService's Self should be mutable to maintain state over incoming requests
2. Request handler is called on a body with unexpected length
3. Bytes should be a DTypePointer instead of a List

@lost gorge how urgent/critical are these? Are you still able to achieve your goals with Lightbug without them? Thinking which ones I should prioritize

[lost gorge]

that's awesome @inland scaffold. Descriptions are capturing the ideas well. (2) is the most critical and urgent one as it actually stops me from being able to use Mocodes in my application and I don't have enough HTTP/sockets knowledge myself to go fix it. Having said that, I took a stab at (1) myself by simply changing the HTTPService trait func(self, ...) to func(inout self,...) but then ran into issues with rValue and mutability in server.mojo which I did not know how to circumvent.

[inland scaffold]

Got it. Can you maybe attach a code snippet inside the issue #2 with the service code/request payload you're using so I can try and reproduce the problem on my end?

#1 should be fixable, I'll look into it

[lost gorge]

Done !

[inland scaffold]

Thank you, will see if I can fix this today. I'll keep you posted

[inland scaffold]

@lost gorge couldn't reproduce yet, asked a couple questions in the issue. If you can let me know the OS you're using that would also help 🙏

[inland scaffold]

@lost gorge (2) should be good now as long as a Content-Length header is set (which is the case in your code). Will try (1) next

[lost gorge]

hi @inland scaffold I confirm (2) is fixed also on my system with those fixes you made. Many thanks !!

[lost gorge]

Hello @inland scaffold , me again 😉 So I have finally found some time to do speed benchmarking with the simplest possible lightbug server that mimicks the behaviour I ultimately need from my lightbug version of the decoder This simple lightbug server just sends back the received packet wrapped in an OK() message. I have a test client that sends packets in a loop to this server, and it measures the elapsed time. What I found is as I increase the size of the packet, with a lightbug server the packet rate decreases rapidly whereas with a flask or fastapi python server that does the same job, the packet rate decreases much less. Again I don't want to just go ahead and create a GIT issue for this as I realise that large binary HTTP packets (up to several 100kByte) were probably not what you had in mind for lightbug. Please advise.

[inland scaffold]

@lost gorge in general, performance definitely needs improvement. I suspect that in your particular case this has to do with the fact that there is currently conversion to strings and back going on in various places. I'm actually removing these conversions by default, which should also make it easier to serve binary files like images, see this PR https://github.com/saviorand/lightbug_http/pull/43 . I haven't benchmarked it yet though, maybe we can try it out with your test code?
In this (general) performance improvement issue someone also posted nice flame graphs from perf for Lightbug, those were running an older Lightbug version though https://github.com/saviorand/lightbug_http/issues/6 .
Definitely interested to try and improve performance for your case. I'm a newbie in the performance world though, would appreciate any tips/suggestions

Another issue is the practical absence of async (since there's no runtime for async functions) in Mojo, this is also a drag on performance.
But in your case I think as long as we remove redundant conversions and re-assignments of variables and boil it down to a minimum Mojo layer + external C calls to socket apis we should be able to get close to flask (or better) in terms of performance

[lost gorge]

sounds good. I am happy to post my test client code eg in a github issue that exists or one that you create for it, let me know

[inland scaffold]

Created an issue here, can you share the code? So I can try on my end. Thanks a lot, really appreciate this info

https://github.com/saviorand/lightbug_http/issues/45

@lost gorge I also saw in your repo gRPC support is something that could be interesting? I might create an issue for that as well

[lost gorge]

I would LOVE grpc support over plain HTTP every day, if that is within your roadmap ideas that would be fantastic.

The way I see it the biggest impediment to Mojo adoption is that it is not taking care of IO. Wonderful if you can demonstrate a fast algorithm on a file or data in RAM. Now how do we get that into a microservices oriented ecosystem because no, we don't all want to go rewrite all of the existing code in C, C++ etc in pure mojo. Real programs act on real data coming from the external world and send the result back to the real world, preferably at the same whopping speed of the algorithm itself.

[inland scaffold]

Yes, I'm definitely interested in gRPC support as well. Haven't thought through the logistics yet. Maybe a separate library makes more sense, like lightbug_grpc . To keep things separate

[warped harness]

@inland scaffold Instead of directly building it as a gRPC-specific library, It will be better if we structure it as a general RPC (Remote Procedure Call) library. The gRPC implementation can then be built on top of this general RPC framework. Also another like Cap’n Proto can be built on top of general rpc.

[lost gorge]

@inland scaffold I updated the ticket with test client and server code and corresponding results obtained on my machine. Happy hunting !

[inland scaffold]

@lost gorge thanks! Will give it a go and keep you posted if I can improve something

@warped harness good point, let me know if you wanna collaborate on this, we can coordinate over DMs

[warped harness]

Thank you so much. I am interested. It will be a great opportunity for me to learning new things and gain some skills.

[lost gorge]

hi @inland scaffold I was wondering whether you have made any progress on the speedup exercise for large message sizes ? Are you close to a solution ? If not, I may have to fork and hack/slash myself in order to meet my project deadline

[inland scaffold]

We managed to achieve better performance on this PR https://github.com/saviorand/lightbug_http/pull/50, @arctic cloak helped a lot with that. But now I'm getting some issues with running your test, I think I need to refactor request processing logic. If you can try pulling this and have any ideas on how to debug the errors we're getting would really help!

The only thing is today's hightly of Mojo broke some things again, if you can maybe use yesterday's nightly version

Let me know if you encounter any issues running this

It's still slow on the largest payload in your test , but I have an idea on how to fix that. Will try today

[lost gorge]

Hi @inland scaffold I myself stay on stable builds rather than nightly to avoid unexpected time lost any given day. Eg now I am on v24.3.0. Are you saying your fixes/improvements won't be compatible with that Mojo version ?

[inland scaffold]

Haven't tested it on a stable build yet, I'm developing on nightly

[inland scaffold]

@lost gorge looks like a release is dropping today so the changes from nightly should be incorporated into the stable build

Tested this with 24.4, should work. But I'm still debugging to make your test work, right now breaks after some point. If you have any idea on how to fix let me know https://github.com/saviorand/lightbug_http/pull/50

[lost gorge]

okay @inland scaffold as soon as I have upgraded to 24.4 I will give it a go !

[astral sundial]

Congrats @lost gorge, you just advanced to level 5!

[lost gorge]

I've just given it a try but it seems to crash real quick with my test client indeed. One thing I do notice is that while I am sending "Content-Type': 'application/octet-stream'", the lightbug server reports: "Content-Type: text/plain" and also while I send 1000 byte packet body lightbug server reports "Content-Length: 998". Maybe this is stuff I should comment in the pull request ticket

[inland scaffold]

Yup, thanks. Let's continue there!

[inland scaffold]

Lightbug has reached 420 stars on Gitihub 😏 🚬 🔥

[stray shadow]

Hello,
SSE(server sent event) can be an easy to use alternative to websockets until it gets implemented with async await:
https://www.pubnub.com/guides/server-sent-events/

It basically upgrade an http get request into a single-way stream.
The server first set the header to "text/event-stream.", then the socket is stored into an array.
Whenever the server need to send data, it just write to the socket 👍
On the browser-side, a simple js callback is set for whenever an event is received.

This is particularly useful for 2D games in the browser, but also for your AI ui's !
The advantage is that it has way less complexity than websockets.
If there are 10 visitors, it is as easy as looping an array and sending data (usually json).
Because the socket stays open, it can be used as an session too.

(ping to @inland scaffold)

[inland scaffold]

This is great, thanks a lot! Actually this might be a good idea to implement first before Websockets

[stray shadow]

There is a great test suite to test websocket implementations:
https://github.com/crossbario/autobahn-testsuite/

The list of projects and users that used it is impressive 👍

(ping to @inland scaffold)

[inland scaffold]

Looks cool, will check it out. By the way, if you're interested in contributing to a SSE/websocket implementation let me know, happy to discuss!

[stray shadow]

Working on it 🔥 ! already have the connection upgrade to websocket and the ability to receive messages of all sizes !
It is quite difficult to implement all the features (fragments) and many things could raise.
Let's focus on an example that can do receive and send.

We might need to get on an audio conference and adapt it to lightbug 👍

Documentation:
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#the_websocket_handshake

Would you mind if i PR a websocket.mojo to lightbug in a work_in_progress folder?
(That way you can scavage the example and integrate to lightbug)

[inland scaffold]

Definitely!! Also happy to get on an audio conference. Maybe let's do the PR first, I'll try my best to integrate, and then we can coordinate/talk to resolve any remaining questions

[stray shadow]

🥳 here is the commit: https://github.com/saviorand/lightbug_http/pull/54
(send receive upgrade)

[inland scaffold]

Lightbug 0.1.3 Release just dropped!

Featuring performance and stability improvements and a new installation workflow -- you can now add Lightbug as a dependency to your mojoproject.toml and import Lightbug with Magic to use in your projects.

Check out the README for details on how to try it out:
https://github.com/saviorand/lightbug_http

[inland scaffold]

Lightbug 0.1.4 Release just dropped!

Headers are much more ergonomic in Lightbug 0.1.4 thanks to @rain pelican 's contribution!

There are now three options for specifying the headers that are accepted as input to HTTPRequest or HTTPResponse:

1. Assigning to headers directly:

var res = HTTPResponse()
res.headers["Connection"] = "keep-alive"
res.headers["Content-Type"] = "application/json"

2. Passing one or more instances of the Header struct to Headers:

var header = Headers(Header("Connection", "keep-alive"), Header("Content-Type", "application/json")
var res = HTTPResponse(headers)

3. Using the parse_raw method on Headers:

var headers_str = bytes(
        """GET /index.html HTTP/1.1\r\nContent-Type: application/json\r\nContent-Length: 1234\r\nConnection: keep-alive\r\n\r\n"""
    )
    var header = Headers()
    var b = Bytes(headers_str)
    var reader = ByteReader(b^)
    var method: String
    var protocol: String
    var uri: String
    method, uri, protocol = header.parse_raw(reader)

The headers can then be accessed as header["Content-Type"], "text/html"

The codebase is also much more Pythonic now with refactors from @rain pelican , with more use of dunder methods and direct string operations.

[astral sundial]

Congrats @inland scaffold, you just advanced to level 7!

[tawny jungle]

Do you have some benchmarks?

[inland scaffold]

@tawny jungle yup, some of the latest ones were posted by @rain pelican here https://github.com/saviorand/lightbug_http/pull/61#issuecomment-2362104634

[rain pelican]

FYI I was running it on an M3 chip

[tawny jungle]

Do you plan on having some sort of templates? I wrote a PoC, can be seen here: https://arax.ee/mojo I’d be interested in having something similar to Go.

[inland scaffold]

Do you mean for HTML specifically or something general-purpose like https://pkg.go.dev/text/template ?
For HTML I have some future plans but this will be in a separate library called lightbug_webthat will build on lightbug_http

[tawny jungle]

HTML given the context.

[atomic salmon]

I'm starting to implement a mojo-websockets package. Totally WIP but my intention is to roughly conform the python-websockets one, first starting with the sync version, and later the async one

[inland scaffold]

@atomic salmon we also have an open PR with @stray shadow to add websockets to lightbug, in case you'd like to take a look: https://github.com/saviorand/lightbug_http/pull/57

[atomic salmon]

wow! I did not know that PR! Will take a look. Thanks!

[inland scaffold]

Lightbug 0.1.5 Release just dropped!

The most important contribution is by @rain pelican - the HTTP client now follows redirects.

We've also reorganized the code quite a bit and removed the client and server implementations that were calling into Python for socket interactions.
It's all Mojo now

https://github.com/saviorand/lightbug_http/releases/tag/v0.1.5

[arctic cloak]

Great job with the speed improvements! @inland scaffold @rain pelican

[lost gorge]

hi @inland scaffold , I am curious how you guys will deal with mojo 24.6 and the symbol clash on "write", I think that should hit your fn write() from lightbug's libc.mojo ...

[inland scaffold]

Hi, hmm no idea, didn't look into it yet, going to do that soon probably, along with other 24.6 updates. If you solve this somehow let me know!

[lost gorge]

hi, fyi this is how I solved it in 24.6 making use of mojo FileDescriptor and Span, after a life saving hint from @rich wigeon and then some searching in mojo stdlib open source code to find examples on how to work with Span: fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int:

var fildes_mojo = FileDescriptor(int(fildes))
var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)

fildes_mojo.write_bytes(byte_span)
return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written

As per the comment on last line, I'm not super happy with this as I lost the functionality of returning the amount of bytes actually written. But I verified the functionality is otherwise identical.

[rich wigeon]

The ``` needs to be on its own line.

[lost gorge]

God I am so bad at these editing tricks 🙂 🙂

[rich wigeon]

fn my_posix_write(fildes: c_int, buf: UnsafePointer[Scalar[DType.uint8]], nbyte: c_size_t) -> c_int:

    var fildes_mojo = FileDescriptor(int(fildes))
    var byte_span = Span[Byte, ImmutableAnyOrigin](ptr=buf, length=nbyte)

    fildes_mojo.write_bytes(byte_span)
    return nbyte # this is a bit of a hack we'd really want to return the nr of bytes written 

[rich wigeon]

https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline

[inland scaffold]

Lightbug 0.1.6 Release just dropped!

Featuring many great improvements from @rain pelican and cookie support by @robin-schoch

We also support Mojo 24.6 now 😄
The full changelog, same as on Github:

• Keep persistent connections in client by @bgreni in #69
• add script for server benchmarking by @bgreni in #71
• Add support for chunked transfer encoding by @bgreni in #70
• Feature/47 cookie support by @robin-schoch in #74
• add integration test by @bgreni in #72
• Allow mutation in server-side service implementation by @bgreni in #76
• Catch exceptions from service handler and return internal error by @bgreni in #77
• bump to 24 6 by @saviorand in #78

https://github.com/saviorand/lightbug_http/releases/tag/v0.1.6

[inland scaffold]

There were a couple issues with v0.1.6 so we shipped a hotfix release 0.1.7. Don't forget to update Lightbug's version in the dependencies section of your mojoproject.toml !
https://github.com/saviorand/lightbug_http/releases/tag/v0.1.7

[inland scaffold]

Lightbug 0.1.9 Release just dropped!

This time with some bangers -- including a refactor of how the socket works (it's a struct now) + basic UDP support, both by @arctic cloak !

Full changelog:

• Introduce Socket and refactor connection caching by @thatstoasty in #86
• UDP Socket support by @thatstoasty in #87

https://github.com/Lightbug-HQ/lightbug_http/releases/tag/v0.1.9

[fair bloom]

I've just tried to install a freshly init'd project, with the following mojoproject.toml...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max",
    "https://repo.prefix.dev/modular-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"

But, when I run mojo install, I get the error message: Cannot solve the request because of: No candidates were found for lightbug_http >=0.1.9..

Any idea what the problem is?

UPDATE: I should also add that even if I change the [dependencies] to lightbug_http = ">= 0.1.8" or even ``lightbug_http = ">= 0.1.7"`, I get similar errors...

[arctic cloak]

I don’t think it’s in the modular-community channel yet. But it is in the mojo-community one

[inland scaffold]

Yup, the PR to add Lightbug to modular-community is pending for now

[fair bloom]

Good catch! I changed my mojoproject.toml to...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max",
    "https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "test_lightbug_http"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
max = ">=24.6.0,<25"
lightbug_http = ">=0.1.9"

But I get the same error!

@inland scaffold If I change the dependency to lightbug_http = ">=0.1.8", it installs. Does this mean that the 0.1.9 release is not yet in the mojo-community, as @arctic cloak suggested?

[inland scaffold]

Hmm when I check here it looks like 0.1.9 wasn't published 🤔
https://prefix.dev/channels/mojo-community/packages/lightbug_http

Looks like there was an error in the publishing CI job, but the job didn't fail for some reason, so I assumed everything was fine https://github.com/Lightbug-HQ/lightbug_http/actions/runs/12835368948/job/35794658728#step:3:1

Will fix that first thing tomorrow. Thanks for the catch!

But hopefully you can test with 0.1.8 for now, unless you need UDP 🙂

[fair bloom]

Cool! At least everyone know what the score is 😃 I'll begin testing in earnest tomorrow morning. I can live without UDP for the moment (~ pauses to look up what UDP stands for ~) 😉

[fair bloom]

Just to confirm, the project installs with 0.1.9 now. Thanks 🙏🏽

[inland scaffold]

Yup, just fixed the issue 20 minutes ago!
Looks like a bug in rattler-build. That's what we get for using their binary from the latest release, haha. Pinned it to a version from a couple weeks ago, now it works. On the upside, was able to report the problem to maintainers so hopefully they can resolve it before it affects more people: https://github.com/prefix-dev/rattler-build/issues/1354

[fair bloom]

While you're here, I'm just looking at the README.md and I'm getting confused as to first steps to get something up and running.

I have an empty test project with just the magic.lock and the mojoproject.toml files. The README talks about "Add your handler in lightbug.🔥"... does this mean adding to the lightbug.:fire: that's in the lightbug_http folder, or to a copy in my own project folder?

The you mentions "For example, to make a Printer service that prints some details about the request to console:" and give some mojo code for a printer service, followed by, in step 6, some more mojo code for a server. Do these pieces of code go in separate .mojo units and are they run separately?

Sorry if I seem dumb here! There's probably just one vital thing that I'm missing here, and I'm sure that once I've got a test example working, I'll be able to proceed...

[fair bloom]

Panic over! with the help of this https://hackernoon.com/lightbug-the-first-mojo-http-framework, I got the default page showing...

Presumable the idea is for my own project, to copy the lightbug.🔥 file to my own folder and then build an app from there...

[inland scaffold]

Glad you got it working!
Actually you can use Lightbug's primitives, like Server, HTTPRequest, HTTPResponse, and the HTTPService trait, from anywhere in your code. lightbug.🔥 is just an example.
So as long as you define some services with needed functionality somewhere in your code and start a server, e.g. with server.listen_and_serve(), it can be in any file in your repo

[fair bloom]

tbh, your instructions in the hackernoon article were a little easier than the github README.md to follow, the only problem is that some of the info, I'm guessing, is outdated in this brave new world of Magic we are in 😉

[inland scaffold]

Ah, true. I'll check out the article once more, will maybe borrow something for the README. It's been a while now 😅

[fair bloom]

I've just created this test client.mojo based on your example code...

from lightbug_http import *

@value
struct Printer(HTTPService):
  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    print("Request URI: ", to_string(uri.request_uri))

    var header = req.headers
    print("Request protocol: ", req.protocol)
    print("Request method: ", req.method)
    print(
      "Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
    )

    var body = req.body_raw
    print("Request Body: ", to_string(body))

    return OK(body)

fn main() raises:
  var server = Server()
  var handler = Printer()
  server.listen_and_serve("0.0.0.0:8080", handler)

and I ran it with magic run mojo client.mojo, but in the browser it comes up with Failed to process request which means that the server code is running but presumably has failed in the above code, somewhere. Is there an easy way of debugging this code?

Doh! That's what the print statements are for 😉
For each run, I'm seeing...

Request URI:  /
Request protocol:  HTTP/1.1
Request method:  GET

which suggests it's failing on the...

print(
      "Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
    )

line...

[arctic cloak]

Can you try printing the request object? HTTPRequest implements Writable so you should be able to print it and see what headers you received

Alternatively, Headers is also writable, so you can print req.headers as well

[fair bloom]

This is the contents of req...

Request:  GET / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0

and req.headers...

Request Headers:  host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=0, i
content-length: 0

[fair bloom]

The code is failing on...

print(
      "Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE])
    )

because headers does not contain a dict entry of ``"content-type"`!

[inland scaffold]

Oh wow, does it work if you comment this out?

[fair bloom]

Yes it does!

[inland scaffold]

Okay, so here the Printer service raises an exception at that point when a non-existent key is being accessed, and this "Failed to process request" is how we log these kinds of service errors for now. We might improve that in the future e.g by making the HTTPService return either a response or an error.
For now I would suggest to put things that could throw into a try/except block so you could catch an process the errors in your handler when they happen

[fair bloom]

@value
struct Printer(HTTPService):
  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    print("Request URI: ", to_string(uri.request_uri))

    var header = req.headers
    print("Request protocol: ", req.protocol)
    print("Request method: ", req.method)

    try:
      print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
    except e:
      print("Request Content-Length: Not found")

    var body = req.body_raw
    print("Request Body: ", to_string(body))
    var html = Html()
    html.html()
    html.body("white", "", "", "", "", "", "black")
    html.title("Test")
    html.h1("Test Heading 1")
    html.h2("Test Heading 2")
    html.h3("Test Heading 3")
    html.h4("Test Heading 4")
    html.end_body()
    html.end_html()
    return OK(str(html), "text/html")
    # (body)

fn main() raises:
  var server = Server()
  var handler = Printer()
  server.listen_and_serve("0.0.0.0:8080", handler)

which gives...

.
I found some 25 year old Delphi code of mine for dynamically writing HTML, and I've been converting it 😉

[inland scaffold]

Crazy! 😁

[fair bloom]

Can you think of a way that I can get my test app to dynamically generate images? atm my code has html.image("/images/earlyspring.png") which is generating the HTML code... <img src="/images/earlyspring.png" align="left" border="0"> but the browser is not loading the image fie because, I guess, it has no way of knowing where the file is, because there is no static root directory for the site.

[inland scaffold]

Did you try out this example from the README? It should be possible to create e.g a static folder of your own, then use Mojo's filesystem functions, e.g read_bytes() to serve a file from that folder on a path. And then you could just refer to that file in your HTML. That's basically how the welcome example works

https://github.com/Lightbug-HQ/lightbug_http?tab=readme-ov-file#serving-static-files

[fair bloom]

I did see that, but it seemed to be slightly different from my case. I'm building the HTML as a long string and then passing it to the result of the Printer func...

@value
struct Printer(HTTPService):
  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    print("Request URI: ", to_string(uri.request_uri))

    var header = req.headers
    print("Request protocol: ", req.protocol)
    print("Request method: ", req.method)

    try:
      print("Request Content-Type: ", to_string(header[HeaderKey.CONTENT_TYPE]))
    except e:
      print("Request Content-Length: Not found")

    var body = req.body_raw
    print("Request Body: ", to_string(body))
    var html = Html()
    html.html()
    html.head()
    html.title("Test")
    html.end_head()
    html.body("white", "", "", "", "", "", "black")
    html.h1("Test Heading 1")
    html.h2("Test Heading 2")
    html.h3("Test Heading 3")
    html.h4("Test Heading 4")
    # html.image("/images/earlyspring.png")
    html.para(html.lorem())
    html.para(html.post_modern())
    html.end_body()
    html.end_html()                      # html contains the <img src="/images/earlyspring.png" align="left" border="0"> tag
    return OK(str(html), "text/html")
    # (body)

fn main() raises:
  var server = Server()
  var handler = Printer()
  server.listen_and_serve("0.0.0.0:8080", handler)

[fair bloom]

UPDATE: I think I get it now. The script is called separately for the main body of the HTML and for the image. Is this how my code should look?

@value
struct Printer(HTTPService):
  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    if uri.path == "/":
      var body = req.body_raw
      print("Request Body: ", to_string(body))
      var html = Html()
      html.html()
      html.head()
      html.title("Test")
      html.end_head()
      html.body("white", "", "", "", "", "", "black")
      html.h1("Test Heading 1")
      html.h2("Test Heading 2")
      html.h3("Test Heading 3")
      html.h4("Test Heading 4")
      html.image("/earlyspring.png")
      html.para(html.lorem())
      html.para(html.post_modern())
      html.end_body()
      html.end_html()
      return OK(str(html), "text/html")
    if uri.path == "/earlyspring.png":
      with open("static/earlyspring.png", "r") as f:
        image = f.read_bytes()
      return OK(image, "image/png")
    return NotFound(uri.path)

[fair bloom]

UPDATE 2: Yes! It works! Thank you so much 🙏🏽

[inland scaffold]

yes, perfect! Sorry, just saw this. Let me know if you have any other questions later 🙂

[fair bloom]

Thanks! I think that after my initial confusion, my understanding has reached a kind of critical mass, so hopefully I'll be able to work out any questions I have, but, yes, if I get really stuck, I'll give you a shout 📣

[inland scaffold]

Deal 🤝

[fair bloom]

Hello! Is there any reason why, if I point mymojoproject.toml at ``"https://conda.modular.com/max", my project builds, but if I point it at"https://conda.modular.com/max"`, my project fails to build with the folllowing errors...

~/code/mojo/ca_web  magic run default
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import *
     ^
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:5:16: error: 'Printer' has no 'HTTPService' member
struct Printer(HTTPService):
               ^~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:45:16: error: use of unknown declaration 'Server'
  var server = Server()
               ^~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[1 - ERROR][Thu Jan 23 2025  6:54pm (GMT+0000)]  (main) [*]   took 4s

[inland scaffold]

Not sure what's the difference between these two Max conda URLs 🤔 They look the same?

[fair bloom]

Sorry! I forgot to amend one of them 😮

All I know is that if I have my mojoproject.toml set to...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max",
    "https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"

but, if I set it to...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max-nightly",   # THE ONLY CHANGE
    "https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"

It fails to build with those errors posted above...

[arctic cloak]

Mojo versions become incompatible fairly quickly

In my experience it was like every 1-3 nightly versions. So, trying to maintain a nightly version of any lib is quite time consuming.

[astral sundial]

Congrats @arctic cloak, you just advanced to level 9!

[rich wigeon]

Yup, that's why none of mine are open right now, I can't actually support them.

But I need stuff from nightly.

[inland scaffold]

We have a nightly version, but I haven't updated it for a while. Might ship a new one this week, will let you know

[arctic cloak]

@rich wigeon If you're open to sending out invites, I'd love to see what you're working on!

[rich wigeon]

Do you have 5 servers?

Most of my work is distributed DBs, and networking between them.

No portability yet, so a CX6, CX7, CX8, or Bluefield is currently min spec NIC for this one.

The other libs I have are also probably not super useful, since they assume you've decided that Linux's network stack is too bloated for your app.

And I'm not sure two of them would run on anything aside from my version of fedora on a zen 4 CPU.

[astral sundial]

carlcaulkett has been warned

Reason: Zalgo usage

[fair bloom]

It seems that this is a known bug or inconsistency with nightly⁠... #mojo-bot-help message

[arctic cloak]

At most I have 3 servers I use for my k8s cluster haha. Definitely not in the distributed DB realm outside of work stuff

[rich wigeon]

And I'm guessing "I own the whole server" is not an attitude you would want.

DBs are one of the last applications allowed to do that, so the DB actually tunes the server and shoves everything else onto a single CPU core.

[inland scaffold]

I mean in general it makes sense that some errors pop up, main branch of Lightbug / latest release tag only works on the latest stable version of MAX/Mojo.

[fair bloom]

I guess I'm going to have to stick with stable until it gets sorted. This didn't help 🤣

[fair bloom]

@inland scaffold I'm making a start on CSS...

  fn get_page_html(mut self) -> String:
    var page = Html()
    var style = Style()

    style.p().color("purple")
    style.h1().color("red")
    style.h2().color("yellow")
    style.h3().color("green")

    page.html_head("lightspeed_http and ca_web test", style)
    page.body("grey", "", "", "", "", "", "")
    page.set_font("Arial")
    page.para("", "datetime")
    page.script('let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;')
    page.h1("Test Heading 1").h2("Test Heading 2").h3("Test Heading 3").h4("Test Heading 4").h5("Test Heading 5").h6("Test Heading 6")
    page.end_font()
    page.image("/earlyspring.png")
    page.set_font("Arial", 5, "Teal")
    page.para(page.lorem())
    page.para(page.post_modern())
    page.input_text("password", "1234go", 23, 23)
    page.input_text("password", "1234go", 23, 23, True)
    page.end_font()
    page.end_body()
    page.end_html()
    page.prettify()
    return str(page)

this generates...

[inland scaffold]

Beautiful 🤌

[fair bloom]

I'm not claiming that the visuals are anything to write home about, but the fluent nature of the API is coming on quite nicely 😉

[inland scaffold]

I'm not a design specialist — nice that it works!

[fair bloom]

Google Fonts FTW 😉

fn get_page_html(mut self) raises -> String:
    var page = Html()
    var style = Style()

    _ = style.p().
      font_family("arial").
      color(Colors.blueviolet).
      background_color(Colors.yellow)
    _ = style.h1().
      font_family(GoogleFonts.Audiowide).
      color(Colors.red).
      background_color(Colors.lightblue)
    _ = style.h2().
      font_family(GoogleFonts.Sofia).
      color(Colors.goldenrod).
      background_color(Colors.lightgreen)
    _ = style.h3().
      font_family(GoogleFonts.Trirong).
      color(Colors.green).
      background_color(Colors.lightcoral)
    _ = style.body()
      .background_color(Colors.azure)
      .font_size(16, FontUnit.PX)
      .font_family("Arial, sans-serif")
      .color(Colors.darkblue)

    _ = page.html_head("lightspeed_http and ca_web test", style)
    _ = page.para("", "datetime")
    _ = page.script('let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;')
    _ = page.h1("Test Heading 1").h2("Test Heading 2").h3("Test Heading 3").h4("Test Heading 4").h5("Test Heading 5").h6("Test Heading 6")
    _ = page.image("/earlyspring.png")
    _ = page.para(page.lorem())
    _ = page.para(page.post_modern())
    _ = page.input_text("password", "1234go", 23, 23)
    _ = page.input_text("password", "1234go", 23, 23, True)
    _ = page.end_html()
    _ = page.prettify()
    return str(page)

Here's the HTML source...

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset='utf-8'>
    <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Audiowide|Sofia|Trirong">
    <style>
      p {
        font-family: arial;
        color: blueviolet;
        background-color: yellow;
      }
      h1 {
        font-family: Audiowide;
        color: red;
        background-color: lightblue;
      }
      h2 {
        font-family: Sofia;
        color: goldenrod;
        background-color: lightgreen;
      }
      h3 {
        font-family: Trirong;
        color: green;
        background-color: lightcoral;
      }
      body {
        background-color: azure;
        font-size: 16.0px;
        font-family: Arial, sans-serif;
        color: darkblue;
      }
    </style>
    <title>lightspeed_http and ca_web test</title>
  </head>
  <body>
    <p id='datetime'/>
    <script>
      let datetime = new Date(); document.getElementById("datetime").innerHTML = datetime;
    </script>
    <h1>Test Heading 1</h1>
    <h2>Test Heading 2</h2>
    <h3>Test Heading 3</h3>
    <h4>Test Heading 4</h4>
    <h5>Test Heading 5</h5>
    <h6>Test Heading 6</h6>
    <img src="/earlyspring.png" align="left" border="0">
    <p>lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diem nonummy nibh euismod tincidunt ut lacreet dolore magna aliguam erat volutpat. ut wisis enim ad minim veniam, quis nostrud exerci tution ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. duis te feugifacilisi. duis autem dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit au gue duis dolore te feugat nulla facilisi. ut wisi enim ad minim veniam, quis nostrud exerci taion ullamcorper suscipit lobortis nisl ut aliquip ex en commodo consequat.</p>

    <p>Latin-looking words arranged in nonsensical sentences, set in columns to give the appearance of text on a page. Dummy text is used by the designer to help approximate the look of a page at a stage in the design process before the written text has been received. This way the designer is able, in a very real sense, to shape the material presence of words before they are written and before their meaning is known. Conventionally, due to constraints of time, ability, budget, and habit, the designer is not the author. So conventionally, the student of typography is not encouraged to write (or even read prepared copy) which would waste valuable time spent getting to grips with the mechanics of text layout. Such established working/teaching methods increase the danger of the typographer becoming detached from the meaning of texts. The treatment of copy in purely formal terms, reduced to blocks of texture on a page, has lead to the typographer's obsession with craft and disregard of meaning. Dummy text is text that is not meant to be read, but only looked at; a shape. The choice of Latin is crucial to this function in that it is a dead language. Working with dummy text, the designer only brings into play part of his/her array of tools/skills to convey meaning.</p>
    <input type=text name="password" value="1234go" size="23" maxlength="23" >
    <input type=password name="password" value="1234go" size="23" maxlength="23" >
  </body>
</html>

[inland scaffold]

We're getting close to being able to publish a blog written in Mojo!

[fair bloom]

I'm moving more of the visuals away from the HTML into CSS...

Now including standard fonts and all 1799 Google fonts 😉

  fn get_page_html(mut self) raises -> String:
    var page = Html()
    var style = Style()

    _ = style.
      image_style(Class.round_image).
      width(150).
      height(150).
      border(20, "solid", Colors.darkblue).
      border_radius(75)

    _ = style.
      input_style(Class.fancy_input).
      padding(10).
      margin(5).
      border(2, "dotted", Colors.blue).
      border_radius(5)

    _ = style.p().
      font_family("arial").
      color(Colors.blueviolet).
      background_color(Colors.yellow)

    _ = style.set_h_scale_factor(2)

    _ = style.
      h1().
      font_family(GoogleFonts.Audiowide).
      color(Colors.red).
      background_color(Colors.lightblue)

    _ = style.
      h2().
      font_family(GoogleFonts.Sofia).
      color(Colors.goldenrod).
      background_color(Colors.lightgreen)

    _ = style.
      h3().
      font_family(GoogleFonts.Trirong).
      color(Colors.green).
      background_color(Colors.lightcoral)

    _ = style.
      h4().
      font_family(GoogleFonts.Aclonica).
      color(Colors.blueviolet).
      background_color(Colors.lightblue)

    _ = style.
      h5().
      font_family(GoogleFonts.Bilbo).
      color(Colors.crimson).
      background_color(Colors.lightgreen)

    _ = style.
      h6().
      font_family(GoogleFonts.Salsa).
      color(Colors.black).
      background_color(Colors.lightcoral)

    _ = style.
      body()
      .background_color(Colors.azure).
      font_size(16, FontUnit.PX).
      font_family("Arial, sans-serif").
      color(Colors.darkblue)

    _ = style.
      id(id.lorem).
      font_family("Times New Roman, serif", ).
      color(Colors.chartreuse).
      background_color(Colors.darkblue).
      margin_top(0).margin_bottom(0).
      padding(10).
      font_size(110, FontUnit.PERCENT)

    _ = style.
      id(id.post_modern).
      font_family("Futura, sans-serif").
      color(Colors.gainsboro).
      background_color(Colors.darkblue).
      margin(0).
      padding(20)

    _ = style.
      id(id.datetime).
      color(Colors.darkblue).
      background_color(Colors.lightblue).
      margin(0).
      padding(20).
      font_size(20, FontUnit.PX)

    _ = page.html_head("lightspeed_http and ca_web test", style)
    _ = page.para("", id.datetime)
    _ = page.script(id.datetime, 'new Date(); document.getElementById("datetime").innerHTML = datetime;')
    _ = page.
      h1(GoogleFonts.Audiowide).
      h2(GoogleFonts.Sofia).
      h3(GoogleFonts.Trirong).
      h4(GoogleFonts.Aclonica).
      h5(GoogleFonts.Bilbo).
      h6(GoogleFonts.Salsa)
    _ = page.image("/earlyspring.png", Class.round_image)
    _ = page.para(page.lorem(), id.lorem)
    _ = page.para(page.post_modern(), id.post_modern)
    _ = page.input_text(id.username, "carl", Class.fancy_input, 23, 23, False)
    _ = page.input_text(id.password, "1234go", Class.fancy_input, 23, 23, True)
    _ = page.end_html()
    page.prettify()
    return str(page)

[fair bloom]

@inland scaffold I'm starting to investigate how to handle subitted forms. My understanding from 20 years ago is that the form will be submitted using a "POST" method, as opposed to a "GET" method, but the question then is how can read the submitted for, in particular the input box with the id "username".

My code editor's AI is suggesting this amendment to the func methods...

  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    if uri.path == "/":
       if req.method == "GET":
        return OK(self.get_page_html(), "text/html")
       elif req.method == "POST":
         var username_option = req.form_data.get("username")
         if username_option:
            return OK(self.get_page_html(username_option.value()), "text/html")
         else:
          return OK(self.get_page_html(""), "text/html")
    if uri.path.endswith(".png"):
      return OK(self.get_image(uri.path), "image/png")
    return NotFound(uri.path)

but it seems to be hallucinating HTTPRequest.form_data 😮

Is there a way I can use HTTPRequest.get_body() and parse the raw data?

[fair bloom]

I've just tried it...

  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    if uri.path == "/":
       if req.method == "GET":
        return OK(self.get_page_html(), "text/html")
       elif req.method == "POST":
         var raw_date = String(req.get_body())
         print(raw_date)
         return OK(self.get_page_html(), "text/html")
    if uri.path.endswith(".png"):
      return OK(self.get_image(uri.path), "image/png")
    return NotFound(uri.path)

and look what we get... 😃

🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...

username=carl&password=1234

I think i can work with that 😉

[inland scaffold]

Yes, that's great! Here's a nice instruction from Mozilla on the topic: https://developer.mozilla.org/en-US/docs/Learn_web_development/Extensions/Forms/Sending_and_retrieving_form_data

[fair bloom]

Hi @inland scaffold! Are you around?

[inland scaffold]

Hi, yup I'm here

[fair bloom]

I think there may be a problem with HTTPRequest.get_body() in request.mojo. It seems to be deleting the last 2 characters of what ever is in the page. You know that thing I posted yesterday...

🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...

username=carl&password=1234

Well, the page it came from had the content...

    <form name=form action=/ method=post>
      <input type=text name="username" value="carl" class="fancy_input" size="23" maxlength="23" >
      <input type=password name="password" value="1234go" class="fancy_input" size="23" maxlength="23" >
      <input type="submit" value="Submit">
    </form>

In other words, the password value of "1234go" is being trancated to "1234". I've checked my code and I've examined your code as far as the body_raw property, but I cannot find the missing characters!

Any ideas?

[inland scaffold]

Ah yes, could be the way we deal with null terminators. There are some inconsistencies with the way null terminators work when dealing with strings/bytes in Mojo, I thought we have addressed that, but there was a lot of refactoring recently, could be that there's a regression and it's broken again. Is this version 0.1.9 ? I'll create an issue and investigate

[fair bloom]

Yes, it is 0.1.9.

I did notice that the request body had a coupleof extra bytes at the start which I'm getting rid of with this code...

from collections.dict import Dict

@value
struct PostResponse():
  var _raw_data: String

  fn __init__(out self):
    self._raw_data = String()

  fn __init__(out self, raw_data: String):
    self._raw_data = raw_data

  fn build_dict(self) raises -> Dict[String, String]:
    var dict = Dict[String, String]()
    var raw_data = self._raw_data
    if raw_data == "":
      return dict
    if "&" not in raw_data:
      return dict
    if raw_data.startswith("\r\n"):
      raw_data = self._raw_data[2:]
    var lines = raw_data.split("&")
    for line in lines:
      if "=" not in line[]:
        continue
      var parts = line[].split("=")
      if len(parts) == 2:
        dict[parts[0]] = parts[1]
    return dict

  fn get(self, key: String) raises -> String:
    try:
      var dict = self.build_dict()
      if key in dict:
        return dict[key]
      return ""
    except:
      return ""

  fn dict(self) raises -> Dict[String, String]:
    return self.build_dict()

At first i though that the problem was due to my fix, largely because if also dealt with a 2 character difference!

[inland scaffold]

Are you using some conversion methods in your code elsewhere, like as_bytes() ?

[fair bloom]

No, as_bytes is not is not in my code at all. Anything else I should be looking for?

[inland scaffold]

Is this the handler? When you print raw_date , is it already cut off?

[fair bloom]

With this code...

  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    if uri.path == "/":
      if req.method == "GET":
        return OK(self.get_page_html(), "text/html")
      elif req.method == "POST":
        print(req)
        var post_response = PostResponse(String(req.get_body()))
        return OK(self.get_page_html(post_response), "text/html")
    if uri.path.endswith(".png"):
      return OK(self.get_image(uri.path), "image/png")
    return NotFound(uri.path)

I get...

POST / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
referer: http://0.0.0.0:8080/
content-type: application/x-www-form-urlencoded
content-length: 29
sec-gpc: 1
connection: keep-alive
upgrade-insecure-requests: 1
priority: u=4


username=carl&password=1234

The two spaces before username=carl&password=1234 are, I guess, because of the \r\n being added...

[stray shadow]

Hello, simple checkbox with emoji's if needed:

var tmp_autoplay = "✅ AutoPlay" if PlaySoundsSequence else "⬜ AutoPlay"

(ping to @fair bloom)

[inland scaffold]

Do I understand correctly, this is happening on print(req)? Or are the logs from someplace else

[fair bloom]

Yes! That log two messages ago was from the print(req)!

What was that checkbox stuff all about?

[inland scaffold]

So it must be this line in our code then , will try to reproduce on my end.
If you print req.get_body() or req.body_raw it's the same issue?

[fair bloom]

  fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
    var uri = req.uri
    if uri.path == "/":
      if req.method == "GET":
        return OK(self.get_page_html(), "text/html")
      elif req.method == "POST":
        print("req: HTTPRequest: " + str(req))
        print("req.get_body(): " + str(req.get_body()))
        print("req.body_raw: " + String(req.body_raw))
        var post_response = PostResponse(String(req.get_body()))
        return OK(self.get_page_html(post_response), "text/html")
    if uri.path.endswith(".png"):
      return OK(self.get_image(uri.path), "image/png")
    return NotFound(uri.path)

req: HTTPRequest: 

POST / HTTP/1.1
host: 0.0.0.0:8080
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:135.0) Gecko/20100101 Firefox/135.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-GB,en;q=0.5
accept-encoding: gzip, deflate
content-type: application/x-www-form-urlencoded
content-length: 29
origin: http://0.0.0.0:8080
sec-gpc: 1
connection: keep-alive
referer: http://0.0.0.0:8080/
upgrade-insecure-requests: 1
priority: u=0, i


username=carl&password=1234

req.get_body():
username=carl&password=1234

req.body_raw:
username=carl&password=123

The last one is even weirder, and is not a typo on my part 😉

[fair bloom]

If you want to try the code, it's at https://github.com/carlca/ca_web

[arctic cloak]

The last one is truncated because body_raw is a list of bytes that are not null terminated. So you lose the last character

Usually if I want to print bytes I’ll make it a stringslice

If you repr the request, are there 3 \r\n before the request body?

[fair bloom]

I'm trying to make it a stringslice...

        print("req.body_raw: " + StringSlice.from_utf8(req.body_raw))

but I get the error...

mojo: 'StringSlice[?]' value has no attribute 'from_utf8'

My code was based on a suggestion from kapa ai...

[fair bloom]

What do you mean by repr the request?

[inland scaffold]

Maybe this https://docs.modular.com/mojo/stdlib/builtin/repr/repr/

[inland scaffold]

can also try this? StringSlice(unsafe_from_utf8=req.body_raw)

[arctic cloak]

I was mistaken, HTTPrequest does not implement repr at the moment, but I'm trying to run print(repr(req.get_body())) to check on the newlines present. I ran it with no body and it looks fine, I see 2 CRLF (\r\n\r\n) which is correct, but I need to test it with a request that has a body. I don't think we have an unit tests for request parsing that includes a body yet.

I think I fixed the bug, I will open a PR for it

We should check if content-length > 0 before trying to read the body, and we should also skip any leading \r\n before reading the body. There was a leading \r\n that consumed 2 bytes that was expected in the body itself, so that's why you saw go truncated off the end.

[fair bloom]

Still getting username=carl&password=1234 - missing the final "go" characters.

[fair bloom]

Aha! I had noticed the leading \r\n in the body earlier this afternoon, and took steps to remove them...

from collections.dict import Dict

@value
struct PostResponse():
  var _raw_data: String

  fn __init__(out self):
    self._raw_data = String()

  fn __init__(out self, raw_data: String):
    self._raw_data = raw_data

  fn build_dict(self) raises -> Dict[String, String]:
    var dict = Dict[String, String]()
    var raw_data = self._raw_data
    if raw_data == "":
      return dict
    if "&" not in raw_data:
      return dict
    if raw_data.startswith("\r\n"):    # <== FIX HERE
      raw_data = self._raw_data[2:]
    var lines = raw_data.split("&")
    for line in lines:
      if "=" not in line[]:
        continue
      var parts = line[].split("=")
      if len(parts) == 2:
        dict[parts[0]] = parts[1]
    return dict

  fn get(self, key: String) raises -> String:
    try:
      var dict = self.build_dict()
      if key in dict:
        return dict[key]
      return ""
    except:
      return ""

  fn dict(self) raises -> Dict[String, String]:
    return self.build_dict()

but I didn't think to compensate at the other end. It did seem like a bit too much of a coincidence, though 😉

Do I have to wait for @inland scaffold to accept the PR or is there a fix I can apply in the meantime. I still fear the intricacies of Git - I try to avoid things like PRs, branches and checkouts 😮

[fair bloom]

Anyhow, thanks for your help on this, both of you 🙏🏽. I'll be waiting for the nod to update 😉

[stray shadow]

There is github desktop and sublime merge for easy gui git works if needed

[inland scaffold]

I'll ship a new release today, there are some other updates we wanted to get in anyway

[fair bloom]

Cool! Ready when you are 😉

[fair bloom]

Thanks! I'm actually using Gitkraken on the Mac, and it's getting easier, slowly, as I tentatively investigate more options 😉

[fair bloom]

@inland scaffold! A quick question, if I may... If I want to use an external style sheet, let's call it style.css andI have it stored in my ./static folder alongside my EarlySpring.png image file, how should I refer to it in my HTML which is being served by lightbug_http?

[inland scaffold]

Sure, same approach!

@value
struct Welcome(HTTPService):
    fn func(mut self, req: HTTPRequest) raises -> HTTPResponse:
        if req.uri.path == "/":
            with open("static/lightbug_welcome.html", "r") as f:
                return OK(f.read_bytes(), "text/html; charset=utf-8")

        if req.uri.path == "/logo.png":
            with open("static/logo.png", "r") as f:
                return OK(f.read_bytes(), "image/png")
            
        if req.uri.path == "/styles.css":
            try:
                with open("static/styles.css", "r") as f:
                    content = f.read_bytes()
                    print("CSS file loaded successfully")  # Debug print
                    return OK(content, "text/css")
            except:
                print("Error loading CSS file")  # Debug print

        return NotFound(req.uri.path)

Just need to include it in your HTML like so:

    <link rel="stylesheet" href="/styles.css">

[inland scaffold]

Lightbug 0.1.10 Release just dropped!

With amazing first contributions from @slow ridge and lots of refactoring/stability improvements!

Full changelog:

• Parse URI queries in to key value pairs by @izo0x90 in #237
• Adds url escape sequence decoding by @izo0x90 in #238
• Fix socket by @thatstoasty in #235
• Fix reader not skipping over all CRLF by @thatstoasty in #241
• Process localhost in parse_address by @saviorand in #239

https://github.com/Lightbug-HQ/lightbug_http/releases/tag/v0.1.10

[fair bloom]

I'm happy to confirm that my missing two characters have returned to me 😃. Thanks you @inland scaffold and @arctic cloak 🙏🏽

[inland scaffold]

Yay, the Return of the Bytes

[fair bloom]

Thanks for that code. It was the missing part of the puzzle that has been eluding me all day. It's now enabled me to switch between embedding the style css within the page or referencing it as a separate file...

    var use_static_css = False

    if use_static_css:
      _ = style.save_to_file("static/style.css")
      _ = style.clear()
      _ = page.html_head("lightspeed_http and ca_web test", "/style.css", style)
    else:
      _ = page.html_head("lightspeed_http and ca_web test", "", style)

I've been badgering Google Gemini 2.0 Flash AI a lot today. I think I came close to giving it a nervous breakdown. What didn't help was that my browser Zen decided that today would be a good day to not display the up-to-date source code on "View Page Source". Things improved when I cleared the cache 😉

[stray shadow]

I think it is possible to add an HTTP header that specify that the cache should expire after a certain amount of time 👍

[fair bloom]

👍

[stray shadow]

Yep, see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

[fair bloom]

Hi @inland scaffold! Am I right in thinking that any project using lightbug_http still needs to be build with Max/Mojo Stable as opposed to nightly?

[inland scaffold]

Yup, haven't updated the nightly branch yet. I might get to it today though

[fair bloom]

No worries! In your own time. As long as it's a known issue 😉

[inland scaffold]

@fair bloom actually looks like I can't for now, the nightly branch breaks quite frequently + Lightbug is dependent on other libraries from other authors. So even if I update it now, keeping nightly up-to-date would require all the libraries to commit to updating their codebases every day, which we don't have the collective capacity for right now. Maybe later. I think I'll just close the nightly branch for now.

[fair bloom]

No worries! I guess I'm more concerned about making sure that MY code is fully nightly compatible, whichI guess I can do by temporarily switching to "https://conda.modular.com/max-nightly" tweaking the errors to do with my code only, then switch back to stable.

[fair bloom]

Famous last words 😮 I'm gonna have to stick with stable syntax for now...

[inland scaffold]

Yeah, it's complicated 😅

[fair bloom]

@inland scaffold Check this out...
#mojo message

[inland scaffold]

Nice, looks like you're adding JS?

[fair bloom]

Indeed! This is my get_page_html method now. Featuring a fluent api, static or dynamic CSS, scripting with JS, standard or Google fonts...

  fn get_page_html(mut self, post_response: PostResponse = PostResponse()) raises -> String:
    var page = Html()
    var style = Style()

    _ = style.
      image_style(Class.round_image).
      width(150).height(150).
      border(20, "solid", Colors.darkblue).border_radius(75)

    _ = style.
      input_style(Class.fancy_input).
      padding(10).margin(5).
      border(2, "dotted", Colors.blue).border_radius(5)

    _ = style.p().
      font_family("arial").color(Colors.blueviolet).background_color(Colors.yellow)

    _ = style.set_h_scale_factor(2)

    _ = style.h1().
      font_family(GoogleFonts.Audiowide).color(Colors.red).background_color(Colors.lightblue)

    _ = style.h2().
      font_family(GoogleFonts.Sofia).color(Colors.goldenrod).background_color(Colors.lightgreen)

    _ = style.h3().
      font_family(GoogleFonts.Trirong).color(Colors.green).background_color(Colors.lightcoral)

    _ = style.h4().
      font_family(GoogleFonts.Aclonica).color(Colors.blueviolet).background_color(Colors.lightblue)

    _ = style.h5().
      font_family(GoogleFonts.Bilbo).color(Colors.crimson).background_color(Colors.lightgreen)

    _ = style.h6().
      font_family(GoogleFonts.Salsa).color(Colors.black).background_color(Colors.lightcoral)

    _ = style.body().
      color(Colors.darkblue).background_color(Colors.azure).
      font_size(16, FontUnit.PX).font_family("Arial, sans-serif")

    _ = style.id(id.lorem).
      font_family("Times New Roman, serif", ).font_size(110, FontUnit.PERCENT).
      color(Colors.chartreuse).background_color(Colors.darkblue).
      margin_top(0).margin_bottom(0).padding_top(10).padding_bottom(2).padding_left(10).padding_right(10)

    _ = style.id(id.post_modern).
      font_family("Futura, sans-serif").
      color(Colors.gainsboro).background_color(Colors.darkblue).
      margin(0).padding_top(2).padding_bottom(10).padding_left(10).padding_right(10)

    _ = style.id(id.datetime).
      color(Colors.darkblue).background_color(Colors.lightblue).
      margin(0).padding(0).
      font_size(20, FontUnit.PX)

    var use_static_css = True

    if use_static_css:
      _ = style.save_to_file("static/style.css")
      _ = style.clear()
      _ = page.html_head("lightspeed_http and ca_web test", "/style.css", style)
    else:
      _ = page.html_head("lightspeed_http and ca_web test", "", style)

    _ = page.para("", id.datetime)
    _ = page.script(Script(id.datetime).update_time())

    _ = page.
      h1(GoogleFonts.Audiowide).h2(GoogleFonts.Sofia).h3(GoogleFonts.Trirong).h4(GoogleFonts.Aclonica).h5(GoogleFonts.Bilbo).h6(GoogleFonts.Salsa)
    _ = page.image("/earlyspring.png", Class.round_image)
    _ = page.para(page.lorem(), id.lorem)
    _ = page.para(page.post_modern(), id.post_modern)

    _ = page.form()
    _ = page.input_text(id.username, "carl", Class.fancy_input, 23, 23, False)
    _ = page.input_text(id.password, "1234go", Class.fancy_input, 23, 23, True)
    _ = page.submit()
    _ = page.end_form()

    var post_data = PostData(post_response.get("username"), post_response.get("password"))
    _ = page.para("Username (POST): " + post_data.username, id.username_post)
    _ = page.para("Password (POST): " + post_data.password, id.password_post)

    _ = page.para("Username (DOM): ", id.username)
    _ = page.para("Password (DOM): ", id.password)

    _ = page.button("Update Outputs", id.update_dom)
    _ = page.script(Script(id.update_dom).update_dom(
      (id.username, post_data.username, True),
      (id.password, post_data.password, True)))

    _ = page.end_html()
    page.prettify()
    return str(page)

[fair bloom]

You are welcome to try it out! It's at https://github.com/carlca/ca_web.git.
Just run magic run default.
If you want inline CSS, just change this line (127) in client.mojo from True to False...

    var use_static_css = True

[inland scaffold]

Cool, I'll try this out soon! It's getting pretty feature-rich now 👍

[fair bloom]

You mentioned a while back about hosting with Docker. How complicated is that?

[inland scaffold]

I'm actually currenrly trying to update the dockerfile for Lightbug to work with this: https://github.com/modular/magic-docker/tree/main . Will let you know how it goes

[astral sundial]

Congrats @inland scaffold, you just advanced to level 9!

[fair bloom]

In the meantime, I've been experimenting with Netlify, and I've made some progress! Here is the latest build log...

3:40:49 PM: Netlify Build                                                 
3:40:49 PM: ────────────────────────────────────────────────────────────────
3:40:49 PM: ​
3:40:49 PM: ❯ Version
3:40:49 PM:   @netlify/build 29.58.9
3:40:49 PM: ​
3:40:49 PM: ❯ Flags
3:40:49 PM:   accountId: 5a26827ba6188f059225dccd
3:40:49 PM:   baseRelDir: true
3:40:49 PM:   buildId: 67a4d7def8ab342da742bcf4
3:40:49 PM:   deployId: 67a4d7def8ab342da742bcf6
3:40:49 PM: ​
3:40:49 PM: ❯ Current directory
3:40:49 PM:   /opt/build/repo
3:40:49 PM: ​
3:40:49 PM: ❯ Config file
3:40:49 PM:   No config file was defined: using default values.
3:40:49 PM: ​
3:40:49 PM: ❯ Context
3:40:49 PM:   production
3:40:49 PM: ​

3:40:49 PM: Build command from Netlify app                                
3:40:49 PM: ────────────────────────────────────────────────────────────────
3:40:49 PM: ​
3:40:49 PM: $ curl -ssL https://magic.modular.com/deb11594-7cdd-4b2d-968c-386eb908dd12 | bash && export PATH="$PATH:/opt/buildhome/.modular/bin" && magic project platform add linux-64 && magic run default
3:40:50 PM: Installing the latest version of Magic...
3:40:52 PM: Done. The 'magic' binary is in '/opt/buildhome/.modular/bin'
3:40:52 PM: Could not update shell: nologin
3:40:52 PM: Please permanently add '/opt/buildhome/.modular/bin' to your /opt/build/repo/node_modules/.bin:/opt/build/node_modules/.bin:/opt/node_modules/.bin:/node_modules/.bin:/opt/buildhome/.nvm/versions/node/v18.20.6/bin:/opt/buildhome/.nvm/versions/node/v18.20.6/bin:/home/linuxbrew/.linuxbrew/bin:/opt/buildhome/.swiftenv/bin:/opt/buildhome/.swiftenv/shims:/opt/buildhome/.php:/opt/buildhome/.binrc/bin:/opt/buildhome/.deno/bin:/opt/buildhome/.local/bin:/opt/buildhome/.local/share/mise/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/buildhome/.cask/bin:/opt/buildhome/.dotnet/tools:/opt/buildhome/.dotnet:/opt/buildhome/.cargo/bin to enable the 'magic' command.
3:41:24 PM: ✔ Added linux-64
3:41:32 PM: :fire::bee: Lightbug is listening on http://0.0.0.0:8080
3:41:32 PM: Ready to accept connections...

Look what we have at the bottom! It shows that lightbug_http is running on Netlify. The only problem, I think, is that it needs to be pursuaded that it should serve on https://cawebtest.netlify.app/ rather than http://0.0.0.0:8080. Is this something that can be added in a config?

[inland scaffold]

just need a reverse proxy on top of it, like NGINX. If you can share the config / how you do it I can also try it out?

[fair bloom]

If you want to try it on Netlify yourself, the critical settings are Base directory and Publish directory which are both set to / and the Build command which is curl -ssL https://magic.modular.com/deb11594-7cdd-4b2d-968c-386eb908dd12 | bash && export PATH="$PATH:/opt/buildhome/.modular/bin" && magic project platform add linux-64 && magic run default. Good luck 🤞😉

[inland scaffold]

trying out now

[inland scaffold]

Hmm, apparently Netlify only supports static sites and edge functions, since it's designed for JAMStack. Lightbug's server is not a static site so I'm not sure it's even possible to run it like that 🤔
See here: https://answers.netlify.com/t/support-guide-can-i-run-a-web-server-http-listener-and-or-database-at-netlify/3078

• looks like their Functions only support typescript/javascript/go code and not custom binaries or Docker, see here: https://answers.netlify.com/t/execute-binary-or-run-docker-container/5926

maybe it's easier to just use a cloud provider like https://www.digitalocean.com/ in this case

e.g I know Oracle Cloud has an always-free tier, that's probably useful for testing things out

[rich wigeon]

There's an asterisk on that, you need to rate limit your network bandwidth.

I can hand over the command for that if you go that route.

[fair bloom]

Interesting! I know that Hugo websites are supported, and Hugo is a framework that produced dynamic content written in Go. Here is my site there https://relaxed-williams-d8a311.netlify.app/.

Though I hasten to add that I am speaking from a position of barely contained ignorance on all of the underlying net technologies. What I do know about sockets and all of that jazz stopped being useful in about 2003 😉

[rich wigeon]

Could there be an option for "respond to everything, no matter what address"? That would be nice for local dev, since it would let me hand over a v6 address running on my laptop for coworkers to look at if I'm in an office.

[fair bloom]

I'd be interested in any help with getting on Oracle Cloud. I had an encounter with Digital Ocean in 2018 and that experience rather put me off. The fact that when I went back there just now, the first thing that they seemed to care about was securing my payment details and charging me $5 for nothing,

[rich wigeon]

Everything is fixed-price except for bandwidth. In order to keep bandwidth at the correct amount, you can use tc and the token bucket filter to force a rate limit that will let you use bandwidth in bursts, but will never exceed the free tier. You can use the "burst" argument to ensure you don't have issues with monthly bandwidth resetting.

[inland scaffold]

to make sure I understand, do you mean 0.0.0.0/::, or responding at any path like /, test, etc? or are you talking about NGINX rules?

[rich wigeon]

Accept any IP address, but also ignore the Host header.

Anything which resolves to the right IP address would work.

[fair bloom]

Am I right in thinking that tc etc is a Linux only solution? I'm on macOS!

[rich wigeon]

But the Oracle VM will not be on MacOS.

[fair bloom]

True!!

[fair bloom]

To keep my options open, I went ahead with the Digital Oceans sign up. Despite it quite clearly stating that there would be a $5 charge, to verify my payment capacity, the bastards have just charged me £9.61. Quite how $% in US currency equates to £9.61 sterling escapes me 😡

[astral sundial]

Congrats @fair bloom, you just advanced to level 16!

[rich wigeon]

You are probably paying the currency exchange fee.

[fair bloom]

@rich wigeon I'm nipping at your heals 😉

[rich wigeon]

I think your next target is Jack.

I wonder what I get for level 50. I might have a feature request 😉

[inland scaffold]

Alright, I'll see what I can do. Will keep you posted

[slow ridge]

So (based on this https://docs.netlify.com/frameworks/hugo/) seems like Hugo is a static site gen framework so all the artifacts get created during the build process and then hosted on Netlify ... much like the rest of the frameworks that it (netlify) is compatible with.

@fair bloom from what you shared about your framework it appears that you could def. have it build static sites in the same way that it would produce the markup and scripts for a dynamic site with just the limitation that any values are frozen at build time ... could be a cool feature to have like nextjs etc. ... hosting static sites on netlify build with your own framework would also be pretty sweet.

[fair bloom]

@inland scaffold I've managed to get the Netlify hosted site working https://cawebtest.netlify.app/, but, as you and @slow ridge say, this is a static site in that the content is all generated in the build stage, a concept that I hadn't fully grasped yesterday 😉
The latest code is at https://github.com/carlca/ca_web.git.

It looks as if Netlify won't be able to support my ultimate aim of having a truly dynamic Mojo driven website where the page reacts to user input and generates updated content from the Mojo backend code and updates the screen by manipulating the DOM.

[inland scaffold]

ah cool, and I was able to put the default Lightbug welcome handler behind HTTPS on DigitalOcean via NGINX. Updating the Dockerfile in the repo now to make it easier to reproduce

[fair bloom]

Is that working as a dynamic app or a static app on DigitalOcean?

[inland scaffold]

Dynamic

[fair bloom]

Also, I've found a free web hostings service which, apparently supports dynamic and static web sites. It doesn't have built in support for Mojo, unsurprisingly, so I guess Docker is the way to go here. The service is called Render and can be found at https://render.com/, I found it in this list of "10 Free Web Hosting Solutions for Static and Dynamic Sites" - at https://dev.to/anticoder03/10-free-web-hosting-solutions-for-static-and-dynamic-sites-48g1 (it's number 9).

[fair bloom]

Ooh! Now I'm interested! How much does DO cost, again?

[inland scaffold]

I went with a 1 vCPU / 1GB / 25GB Disk droplet ($6/mo), but that's apparently not enough memory for Docker with Magic, so I also added a 2GB swap file , then it works

[inland scaffold]

Render is pretty well-known, yup

[fair bloom]

@inland scaffold Hi! Did you have any joy investigating Docker with lightpad? I ask because I'm trying to build a barebones Dockerfile...

FROM ghcr.io/modular/magic:latest AS build

COPY ca_web /app
WORKDIR /app

RUN magic run build  # <--- Focus on this failing command

and a mojoproject.toml of...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
  "conda-forge",
  # Having to use Stable for now as the nightly build is giving issues... #mojo-bot-help message
  # "https://conda.modular.com/max-nightly",
  "https://conda.modular.com/max",
  "https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
# platforms = ["osx-arm64"]
platforms = ["linux-aarch64"] # <== Note the change here for Docker/Render
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "*"
lightbug_http = ">=0.1.9"

and it's failing because of...

=> ERROR [4/4] RUN magic run build  # <--- Focus on this failing command                                                                     19.5s
------
 > [4/4] RUN magic run build  # <--- Focus on this failing command:
19.36
19.36   × failed to solve the conda requirements of 'default' 'linux-aarch64'
19.36   ╰─▶ Cannot solve the request because of: No candidates were found for
19.36       lightbug_http >=0.1.9.
19.36
19.36
------
Dockerfile:6
--------------------
   4 |     WORKDIR /app
   5 |
   6 | >>> RUN magic run build  # <--- Focus on this failing command
   7 |
--------------------
ERROR: failed to solve: process "/bin/sh -c magic run build  # <--- Focus on this failing command" did not complete successfully: exit code: 1

Any ideas?

[inland scaffold]

There's only "linux-64" and osx available I think, not "linux-aarch64".
Also, I have a working Dockerfile for the Lightbug itself now under /docker in the repo, feel free to check it out!

[fair bloom]

At one point yesterday,I got the message...

failed to parse project from /app/mojoproject.toml: Expected one of
0.246   │ 'linux-32', 'linux-64', 'linux-aarch64', 'linux-armv6l', 'linux-armv7l',
0.246   │ 'linux-ppc64le', 'linux-ppc64', 'linux-s390x', 'linux-riscv32', 'linux-
0.246   │ riscv64', 'osx-64', 'osx-arm64', 'win-32', 'win-64', 'win-arm64',
0.246   │ 'emscripten-wasm32', 'wasi-wasm32', 'zos-z'
0.246

I'll check out the Dockerfile on your website. Thanks for the heads up.
btw, what do you think of these comments about lightbug? Fair, justified, or completely wide of the mark? #mojo message

[inland scaffold]

It's all true, Lightbug is not ready for production use yet, and we will indeed have a big "rewrite everything" day once async is added to Mojo. But hopefully by the time that's done we're also finished with RFC compliance and features like TLS/HTTP2 support

[rich wigeon]

Remember that you'll also want HTTP/3 support, which involves an entire network protocol.

[rain pelican]

Yeah I was kinda hoping to wait for a QUIC impl to come around so we didn’t have to do that ourselves lol. Plenty of work to be done in 1.1 and 2.0 before we start worrying about that

[rich wigeon]

We might be able to use bindings to Cloudflare's quiche, since it has a C API.

But, Mojo native would be better.

[fair bloom]

@inland scaffold Me again! This code works...

    var server = Server()
    var handler = PageHandler()
    server.listen_and_serve("0.0.0.0:10000", handler)

but this code doesn't...

    var port = getenv("PORT")
    var address = String("0.0.0.0")
    if port.strip() == "":
      address += ":10000"
    else:
      address += ":" + port
    var server = Server()
    var handler = PageHandler()
    server.listen_and_serve(address, handler)

listen_and_serve takes a String type, no? Why is the compiler telling me...

/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:184:28: error: invalid call to 'listen_and_serve': method argument #0 cannot be converted from 'String' to 'StringLiteral'
    server.listen_and_serve(address, handler)
    ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: note: function declared here
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[fair bloom]

Even this doesn't work...

    var server = Server()
    var handler = PageHandler()
    var address = String("0.0.0.0:10000")
    server.listen_and_serve(address, handler)

yet listen_and_serve is defined as...

fn listen_and_serve[T: HTTPService](mut self, address: String, mut handler: T) raises:

The error is...

/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:187:28: error: invalid call to 'listen_and_serve': method argument #0 cannot be converted from 'String' to 'StringLiteral'
    server.listen_and_serve(address, handler)
    ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: note: function declared here
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

What am I missing?

[inland scaffold]

@fair bloom actually I was dumb enough to switch this to a StringLiteral in the latest release, it's already reverted back to String in main, I'll ship a release today so you can use the newest version 😄

[fair bloom]

Just let me know when! If it's within the next 5 hours, great, otherwise I'll look at it tomorrow. In any case, thanks for your efforts 🙏🏽

[inland scaffold]

0.1.11 just in, can you try it out?

[fair bloom]

I'm going in 😉

Looking good! This compiles...

fn main() raises:
  if use_lightbug_http:
    var port = getenv("PORT")
    if port.strip() == "":
      port = "10000"
    var server = Server()
    var handler = PageHandler()
    var address: String = String("0.0.0.0:{}".format(port)) # Explicitly type address as String
    server.listen_and_serve(address, handler)

@inland scaffold While you're around, do you know much about how Dockerfiles work?

[inland scaffold]

I know a thing or two, why? Did you have a question about that

[fair bloom]

Here's my Dockerfile...

FROM ghcr.io/modular/magic:latest

RUN apt-get update && apt-get install -y git

RUN git clone https://github.com/Lightbug-HQ/lightbug_http

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app
# WORKDIR /lightbug_http

ARG DEFAULT_SERVER_PORT=8080
ARG DEFAULT_SERVER_HOST=localhost

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=lightbug.🔥

CMD ["magic", "run", "default"]

You'll notice that I'm pulling in lighybug_http using a git clone. Although I'm setting WORKDIR to /app which contains ca_web, my code, I'm not letting Docker know that lightbug_http is present, as far as I can tell

I did try having WORKDIR /lightbug_http at one point, but that caused all kinds of errors.

As it is, I'm getting these errors...

==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding

I've just tried changing DEFAULT_SERVER_PORT to 10000, since thta seems to be the preferred port for Render to serve through, but that gives the same errors about no ports detected.

Any ideas?

Could it be the use of localhost. Should I set DEFAULT_SERVER_HOST to 0.0.0.0 instead?

[rich wigeon]

Do that. Docker makes a fake network, so it will never see traffic from the primary namespace as localhost traffic.

[fair bloom]

Thanks! Trying it as I type 😉

[inland scaffold]

Might make sense to build and run this Dockerfile locally first, to make sure it works, and then use with the render platform

But yes 0.0.0.0 is a good idea

[fair bloom]

That sounds good. How do I run the Dockerfile locally?

[inland scaffold]

Something like this

1. build the Docker image docker build -t lightbug-app

2. Run the container docker run -p 8080:8080 lightbug-app

Where instead of 8080:8080 it's the port e.g. 10000:10000

[fair bloom]

~/Code/Mojo docker build -t lightbug-app
ERROR: "docker buildx build" requires exactly 1 argument.
See 'docker buildx build --help'.

[fair bloom]

Okay, this is building...
docker build -t lightbug-app -f magic-docker/Dockerfile .

When I ran docker run -p 10000:10000 lightbug-app, nothing visible happened on screen, and nothing was server on http://0.0.0.0:10000/.

[fair bloom]

Did you see what I was asking about the setting of WORKDIR in my Dockerfile?

[fair bloom]

I've rationalised my Dockerfile down to...

FROM ghcr.io/modular/magic:latest

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=lightbug.🔥

CMD ["magic", "run", "default"]

On the basis that the ca_web application, via its mojoproject.toml file, should take care of any dependency on lightpad_http making the RUN git clone https://github.com/Lightbug-HQ/lightbug_http line I had before, superfluous.

But still the Render deploy is failing with...

==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding

I'm setting the environent variable PORT=10000 in the Render settings.

I'll re-read https://render.com/docs/web-services#port-binding and see if anything jumps out at me 🤔

[fair bloom]

The full set of errors is...

==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Out of memory (used over 512Mi)
==> Common ways to troubleshoot your deploy: https://render.com/docs/troubleshooting-deploys

I also tweaked the Dockerfile to...

FROM ghcr.io/modular/magic:latest

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

COPY ca_web /app
WORKDIR /app

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /app/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=client.mojo

CMD ["magic", "run", "default"]

setting the APP_ENTRYPOINT to client.mojo, the main file of my app, rather than lightbug.🔥!

Still the same errors...

[fair bloom]

I've posted a message on the Render forum, but I have the sneaking suspicion that this type of error comes with only having a VM with 512 MB and that you have to shell out $25 per month for a 2 gb VM in order to find out if the thing will work at all.

In the meantime I'm going to investigate Oracle Cloud... tomorrow 😉

[stray shadow]

Hello, good job everybody,
there is a small thing that can help a lot for HTML performance:
when web browser render a page, it then request the icon for the tab,
if the server don't give one, it will request it all the time for all pages,
(so all requests from interaction results in multiple ones)
in mojo ui html, solved it by creating an inlined default one:
example for inlined favicon: https://stackoverflow.com/questions/66935329/use-inline-svg-as-favicon

[inland scaffold]

Maybe try adding some swap space to the machine on render, might need to SSH in there. Here an instruction, although from DigitalOcean https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-20-04

[inland scaffold]

About the workdir, I don't think you need to clone lightbug's repo if Lightbug is a dependency in your project. What you would want to do instead is clone your repo and open e.g the app directory as a workdir, without the COPY directive

[fair bloom]

So...

FROM ghcr.io/modular/magic:latest

RUN apt-get update && apt-get install -y git

RUN git clone https://github.com/carlca/ca_web

# Needed for the 'sed' command below:
ARG DOCKER_PLATFORM=linux-64

WORKDIR /ca_web

ARG DEFAULT_SERVER_PORT=10000
ARG DEFAULT_SERVER_HOST=0.0.0.0

# Replaces the 'platforms = ["osx-arm64"]' with 'platforms = ["linux-64"]' in the 'mojoproject.toml' file:
RUN sed -i "s|^platforms = \[.*\]$|platforms = ['$DOCKER_PLATFORM']|g" /ca_web/mojoproject.toml

EXPOSE ${DEFAULT_SERVER_PORT}

ENV DEFAULT_SERVER_PORT=${DEFAULT_SERVER_PORT}
ENV DEFAULT_SERVER_HOST=${DEFAULT_SERVER_HOST}
ENV APP_ENTRYPOINT=client.mojo

CMD ["magic", "run", "default"]

?

I'll give it a try 😉

UPDATED!

[inland scaffold]

Yup, something like that, although if client.mojo does what I think it does it will not start a server listening on a port. For that you'll probably need a file with your main function where you call listen_and_serve()

[fair bloom]

client.mojo ends with...

fn main() raises:
  if use_lightbug_http:
    var port = getenv("PORT")
    if port.strip() == "":
      port = "10000"
    var server = Server()
    var handler = PageHandler()
    var address: String = String("0.0.0.0:{}".format(port)) # Explicitly type address as String
    server.listen_and_serve(address, handler)

[fair bloom]

Same old story...

==> Starting service...
==> No open ports detected, continuing to scan...
==> Docs on specifying a port: https://render.com/docs/web-services#port-binding
==> Out of memory (used over 512Mi)
==> Common ways to troubleshoot your deploy: https://render.com/docs/troubleshooting-deploys

😦

[rich wigeon]

Mojo pulls in a lot of stuff, I'm not sure 512MB is a good amount, considering that TCP buffers count against you on most platforms, and those need to be fairly large for modern line rates.

[inland scaffold]

Did you add a swap file?

[fair bloom]

I haven't yet. I'm hoping that Oracle Cloud with its 1GB allowance for its free tier will be a bit more forgiving. Besides, from what I can gather, it's very difficult to add a swap file to a Render VM with not much benefits. I'm ready to be persuaded otherwise, of course 😉

[inland scaffold]

I have a 1GB droplet on DigitalOcean and that's not enough either 🤷‍♂️

[rich wigeon]

Oracle allows 24 GB, not 1 GB.

[fair bloom]

Are you talking about disk storage or the VM RAM size?

[rich wigeon]

24GB of memory, and 2 100GB virtual disks.

[fair bloom]

On the free tier? Wow!

[rich wigeon]

4 ARM cores as well.

inbound is free, 10 TB per month outbound is free as well.

If you rate limit the network to 30 Mbps, you'll never see issues.

If you get a bit more fancy, you can probably burst up to gigabit.

Also cert management is free up to a point.

You can technically also have 2 1GB 1/8 core AMD VMs, if you really want x86.

Go to https://www.oracle.com/cloud/free/#free-cloud-trial and check the "always free" tier type.

There's a lot more, because Oracle is desperate to get new customers.

[fair bloom]

Interesting! But I am confused... this is the VM I created yesterday, with only a paltry 1 GB RAM available to my VM. How can I access these other Oracle offers?
Seems I can't upload screenshots to a thread. Here's a link... #mojo message

[fair bloom]

Okay, I've looked at https://www.oracle.com/uk/cloud/free/ and it looks as if I have created a VM in the AMD based Compute VMs with 1/8 OCPU and 1 GB memory each category. You, presumable are talking about the Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs category next to that.

[rich wigeon]

I am talking about the ARM VM.

[fair bloom]

@inland scaffold @rich wigeon Almost there 😃 http://132.145.47.167:10000/ Just a missing image file 😮

Oracle Cloud + ARM, for the win 🏆 Thanks for the suggestion @rich wigeon 🙏🏽

[fair bloom]

The missing image file was caused by that old chestnut, file name case differences between macOS and Linux! Not to mention when I changed the case alone and pushed it to Github, the change did not register; I had to change the file name spelling and then change it back again for Github to take the revised file name 😮

[inland scaffold]

Nice! I don't see anything when I open the page, could be related to an issue I saw myself when testing the Docker setup. I'm currently investigating this

[fair bloom]

@inland scaffold I've just discovered a problem with the latest version of Mojo Stable, 25.1. I get an error... #mojo message

If I set the version in mojoproject.toml with...

[dependencies]
max = "=24.6"
lightbug_http = ">=0.1.9"

it build and runs, but if I have...

[dependencies]
max = "=25.1"
lightbug_http = ">=0.1.9"

then that error, and lots of others suddenly appear 😮

[rain pelican]

oh didn't even realize 25.1 was released yet

[rich wigeon]

You probably need to update lightbug, I imagine that libraries aren't stable between versions.

[inland scaffold]

Looks like it was released today 😄

[inland scaffold]

I just learned that it was released, need to update the library first, might take a little bit of time. Will post an update here when done

[fair bloom]

No worries! I can stick with 24.6 for the moment 🙂

[fair bloom]

In the latest version, the CSS is generated by the Mojo code but is saved as a static resource, keeping the HTML nice and clean 🙂
http://132.145.47.167:8080/

[slow ridge]

Let me know if I can help with any of that, not sure if a few ppl on it will make it go faster or slower 🤣

[fair bloom]

A Zen student went to a temple and asked how long it would take him to gain enlightenment if he joined the temple.
"10 years," said the Zen master.
"Well, how about if I really work hard and double my effort?"
"In that case, 20 years."
😉

[fair bloom]

@rich wigeon I've got a problem with Oracle Cloud and I wonder if you might know the answer. My instance is running and Docker is running and hosting my Mojo app just fine. If I ssh inyo the Oracle Cloud and issue a curl http://localhost:8080 the correct HTML comes back proving that the Mojo app is running.

Where I'm having problems is accessing the page via http://132.145.47.167:8080/. It seems to be okay right at the second, but every time I do an update and push a revised Docker, it seems to take ages before I can gain access to it again.

I wondered if it was down to the firewall provisions. I've added an Ingress rule to allow access on port 8080, and, for a while I was workiing on the assumption that I had to delete the existing Ingress rule and reenter it exactly as before, in order to give the firewall a nudge, as it were, But i've abandoned this theory. Instead, it just seems to take ages before I can access the site again. It's really frustrating 😮

[rich wigeon]

Make sure you have a public address attached. If you have IPv6 at home, that is usually cheaper.

[fair bloom]

That's what http://132.145.47.167:8080/ is, isn't it. To give you an example of the hassle I'm having, that address worked 5 minutes ago, but is not working now!

[rich wigeon]

Huh

I never head these issues.

[inland scaffold]

Yes I think it's a bug in Lightbug ( 😅 ) that I discovered last week. Looking into it currently

[rich wigeon]

I've used that free VM to host a virtual tabletop for DnD for years.

[rich wigeon]

For once, don't blame Oracle I guess.

[inland scaffold]

The server dies after a request due to some issues with processing logic it seems. Interestingly, this neither happens locally, nor in our integration tests. Seems like it happens only when running it in a Docker container

[rich wigeon]

Oh fun!

[inland scaffold]

The error I was getting is

Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read. 
/lightbug_http/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1

This is when just sending a GET from the browser to the running default handler

Must be something straightforward, just haven't had the time to look into it properly yet

[fair bloom]

It's interesting that for a lot of the time that I couldn't access the web page via the address I've just mentioned, I could still ssh into the Oracle Cloud CLI and issue curl http://localhost:8080 and get the full HTML, proving that my app and lightbug were both working as expected. It was just that access to the public web address was being blocked by something, and my first thoughts were that it was firewall related.

[stray shadow]

Could it be a cache thing ?

[fair bloom]

Now it's working again. Go figure 🙃

[fair bloom]

Now it's not working, I've got visions of some vindictive person radomly flicking a switch on and off to block or allow me access 😮

[rich wigeon]

The problem is that you aren't using OracleDB to store things /s

[fair bloom]

Here is my Oracle Cloud CLI...

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED         STATUS                    PORTS                                       NAMES
e8c821b727d2   ghcr.io/carlca/magic-image:latest   "magic run default"   8 seconds ago   Up 7 seconds              0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              42 hours ago    Exited (0) 42 hours ago                                               objective_yonath
[opc@ca-web-arm ~]$

[rich wigeon]

Is the container going down?

[fair bloom]

and here is the HTML from my app..

[opc@ca-web-arm ~]$ curl http://localhost:8080
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset='utf-8'>
    <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Audiowide|Sofia|Trirong|Aclonica|Bilbo|Salsa">
    <link rel="stylesheet" type="text/css" href="style.css">
    <title>lightspeed_http and ca_web test</title>
  </head>
  <body>
    <p id='datetime'></p>
    <script>
      let updateTime = function() {
         var datetime = new Date();
         document.getElementById('datetime').innerHTML = datetime
      }
      setInterval(updateTime, 1000);
    </script>
    <h1>Audiowide</h1>
    <h2>Sofia</h2>
    <h3>Trirong</h3>
    <h4>Aclonica</h4>
    <h5>Bilbo</h5>
    <h6>Salsa</h6>
    <form name=form action=/ method=post>
      <input type=text name="username" value="carl" class="fancy_input" size="23" maxlength="23" >
      <input type=password name="password" value="1234go" class="fancy_input" size="23" maxlength="23" >
      <input type=submit value=Submit>
    </form>
    <p id='username_post'>Username (POST): </p>
    <p id='password_post'>Password (POST): </p>
    <p id='username'>Username (DOM): </p>
    <p id='password'>Password (DOM): </p>
    <button type="button" onclick="updateDom()" >Update Outputs</button>
    <script>
      let updateDom = function() {
         document.getElementById('username').innerHTML += '';
         document.getElementById('password').innerHTML += '';
      }
    </script>
  </body>
</html>

[fair bloom]

No! That's the confusing thing... the container doesn't seem to be going down, just access to it from outside of Oracle, ie. through http://132.145.47.167:8080, seems to be intermittent...

[rich wigeon]

Hmmm

Not sure what the problem is.

[inland scaffold]

do you see the logs in the container? I bet it's the same issue I was having

[fair bloom]

Where do I find them?

[inland scaffold]

If you have SSH access or direct access over from Render to the VM you can do docker ps -a to show all running Docker containers with IDs, and docker logs container_id with the container ID of this container

[fair bloom]

I'm using Oracle Cloud now but yeah, even when I couldn't access my page by the web, docker ps -a was showing...

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED         STATUS                    PORTS                                       NAMES
e8c821b727d2   ghcr.io/carlca/magic-image:latest   "magic run default"   8 seconds ago   Up 7 seconds              0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              42 hours ago    Exited (0) 42 hours ago                                               objective_yonath
[opc@ca-web-arm ~]$

I didn't know about docker logs container-id though... I'lll try that now 😉

Is this what you had?

🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1

I see it is the same 😃

[fair bloom]

It seems strange they way the page is sometimes served but not others. Maybe it's some kind of cumulative problem whereby something has to reach a critical level before an exception is thrown... 🤔

[inland scaffold]

Yup, it's a bug. I'll make sure this is addressed, probably on the weekend, once we switch to the new stable release

Thanks for sharing the details

[fair bloom]

np. Good catch 🫴⚾️

[fair bloom]

@inland scaffold Interesting that sometimes it seems to work okay...

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED              STATUS                  PORTS                                       NAMES
6bf9730aa18a   ghcr.io/carlca/magic-image:latest   "magic run default"   About a minute ago   Up About a minute       0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              2 days ago           Exited (0) 2 days ago                                               objective_yonath
[opc@ca-web-arm ~]$ docker logs 6bf9730aa18a


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$ docker logs 6bf9730aa18a


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$

[fair bloom]

Amazingly, it has been up and working for an hour or more, but I just cannot work out what action will cause it to fail... 🤔

[fair bloom]

@inland scaffold Just woken up, and after leaving it in a working state last night, I tried it again, and found that it was failing, this time with a slightly different message...

🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to read request
ERROR - ReceiveError: An error occurred while attempting to receive data from the socket. Error code: 104
ERROR - Socket.receive: Failed to read data from connection.
ERROR - TCPConnection.read: Failed to read data from connection.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1

[inland scaffold]

104 is "connection reset by peer" I think.. Thanks for sharing, will try to reproduce

[fair bloom]

Hi @inland scaffold! I see that Lightbug is up to v0.1.12 as of 2 days ago. Does this mean that you've sorted out the crashing problem?

[fair bloom]

I guess not...

🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to parse request
ERROR - HTTPRequest.from_bytes: Failed to parse request headers: No more bytes to read.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
[opc@ca-web-arm ~]$

[inland scaffold]

Not yet, this release was for bumping to Mojo 25.1 . But it's not on Modular official channel yet, that's why I haven't announced it so far. The crash is still a work in progress, will post here when it's fixed!

[fair bloom]

No pressure 😉 Good luck 🤞

[inland scaffold]

@fair bloom we got a fix! Going to release it tomorrow. Still not on the modular-community channel though, but I'll try to get it published on the mojo-community one

[fair bloom]

Ooh! How exciting 😃 I look froward to seeing it. Good work, that man 😉

[inland scaffold]

Version 0.1.13 just dropped including this fix, feel free to try it out!

[fair bloom]

I'm just trying to sort out a Scala issue, right now. I'll try to get on to it later this evening. Otherwise it'll be first thing in the morning 😉

[fair bloom]

I'm getting a few build errors...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import *
^
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:49:20: error: 'PageHandler' has no 'HTTPService' member
struct PageHandler(HTTPService):
                   ^~~~~~~~~~~
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:179:18: error: use of unknown declaration 'Server'
    var server = Server()
                 ^~~~~~

What version of Max shouldI be using now?

This is my mojoproject.toml...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
  "conda-forge",
  # Having to use Stable for now as the nightly build is giving issues... #mojo-bot-help message
  # "https://conda.modular.com/max-nightly",
  "https://conda.modular.com/max",
  "https://repo.prefix.dev/mojo-community",
]
description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
# max = "=24.6"
# max = "=25.1"
max = "*"
lightbug_http = "=0.1.13"

[inland scaffold]

25.1 should work 🤔

[fair bloom]

Any chance of looking at https://github.com/carlca/ca_web to see why it isn't building? 🙏🏽

[inland scaffold]

Hmm I will try it out, will let you know how it goes

Thanks for all the feedback! This is really valuable and helpful

[fair bloom]

Did you see this message from @arctic cloak... #mojo message

[inland scaffold]

@fair bloom yup, good to know. small-time is a Datetime package from Mikhail that Lightbug uses under the hood 🙂

[fair bloom]

Ah! Okay, I searched for small-time last night but could only find a crypto package 😉

[fair bloom]

Hey @arctic cloak! Did you have chance to look at the small_time hold up? I'm completely stuck without a fix... 😉 @inland scaffold I assume there's nothing you can do about this problem, or is there? 🙏🏽

[arctic cloak]

I have not yet, but if you run magic list what version of lightbug, max, and smalltime are present in your env?

[fair bloom]

Here you go...

Package               Version       Build                 Size       Kind   Source
lightbug_http         0.1.13        h60d57d3_0            1.3 MiB    conda  lightbug_http
max                   25.1.1        release               9.6 KiB    conda  max
max-core              25.1.1        release               199.9 MiB  conda  max-core
max-python            25.1.1        release               103.4 MiB  conda  max-python
small_time            0.1.6         h60d57d3_0            538.3 KiB  conda  small_time

[arctic cloak]

Ah, max version 25.1.1? And small-time’s latest version is 0.1.8. I pinned it to 25.1.0 only, so that’s why you’re seeing an issue

I will update it to a version range between >25.1.0,<25.2.0 later. In the meantime you should be good if you use 25.1.0?

[arctic cloak]

@inland scaffold I pushed a new version with a looser version pin

[fair bloom]

@inland scaffold I've tried with both max="25.1.0" and max="25.1.1"...

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.13"
# small_time = "=0.1.6"
small_time = "*"

In both cases the result is the same...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

Changing small_time to small_time = "=0.1.8" didn't work...

✔ magic is already up-to-date (version 0.7.0)

  × failed to solve the conda requirements of 'default' 'osx-arm64'
  ╰─▶ Cannot solve the request because of: lightbug_http >=0.1.13 cannot be installed because there are no viable options:
      └─ lightbug_http 0.1.13 would require
         └─ small_time ==0.1.6, for which no candidates were found.



  × failed to solve the conda requirements of 'default' 'osx-arm64'
  ╰─▶ Cannot solve the request because of: lightbug_http >=0.1.13 cannot be installed because there are no viable options:
      └─ lightbug_http 0.1.13 would require
         └─ small_time ==0.1.6, for which no candidates were found.

[inland scaffold]

Hmm yup, I'll need to update the small time dependency in Lightbug for this to work. Will do that shortly

[inland scaffold]

hmm actually main and the latest tag should both be on 0.1.8 already. Can you try removing the .magic folder and magic.lock and running the same command again?

[fair bloom]

With...

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.13"
small_time = "=0.1.8"

I get the same,,,

  × failed to solve the conda requirements of 'default' 'osx-arm64'
  ╰─▶ Cannot solve the request because of: lightbug_http >=0.1.13 cannot be installed because there are no viable options:
      └─ lightbug_http 0.1.13 would require
         └─ small_time ==0.1.6, for which no candidates were found.



  × failed to solve the conda requirements of 'default' 'osx-arm64'
  ╰─▶ Cannot solve the request because of: lightbug_http >=0.1.13 cannot be installed because there are no viable options:
      └─ lightbug_http 0.1.13 would require
         └─ small_time ==0.1.6, for which no candidates were found.

[inland scaffold]

Can you try removing the small time from your mojoproject file? Should pull it anyway because Lightbug depends on it

[fair bloom]

With...

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.13"

it's back to our old friend...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[fair bloom]

Could it be anything to do with the fact that I am using macOS and @toasty, I think, is using Linux. Yes, I know these things should be portable between OS's but when unexpected trailing bytes are mentioned, it automatically raises red, OS shaped flags for me 😉

[fair bloom]

The only other place on the internet that I can find the phrase "unexpected trailing bytes" is here... https://github.com/SmingHub/Sming/issues/2158#issuecomment-736529277

[inland scaffold]

Hmm I'm also on Mac and it works for me locally. Will try to reproduce again tomorrow

[fair bloom]

@inland scaffold I've also git clone'd small-time locally... is there some way I can use it as a local dependency. @arctic cloak Please can this be sorted soon? I am absolutely stuck with this impasse 😮

[inland scaffold]

Didn't have much time to work on this yet, will try again today. I don't know if it helps, but you could also just revert to older working versions of the packages so it's not blocking you

[inland scaffold]

I pushed a new release earlier today, 0.1.14, could you try it out?

[fair bloom]

@inland scaffold Using...

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.14"

I still get...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

I tried deleting .magic and magic.lock like you suggested before, but I still get the same result.

The file /Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg is being copied into place, but something is, clearly, going wrong!

Just a thought, could you not, simply, add the code from small_time into your own lightbug_http units? 🤔

.
.
btw, the date/time stamp of small_timr.mojopkg is 26/12/2024 - 17:35... Does that seem likely given that @arctic cloak has, presumably, worked on it since then?

[arctic cloak]

Yeah, your installation for some reason is pulling a version of small_time that’s old. I’m assuming it’s still 0.1.6, which is from mojo 24.6. If you create a new project and add in lightbug, do you get the same behavior?

[fair bloom]

Curiouser and curiouser! I've created a new project with the following mojoproject.toml...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
  "conda-forge",
  "https://conda.modular.com/max",
  "https://repo.prefix.dev/mojo-community",
]
name = "test_small_time"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]
default = "magic run mojo hello.mojo"

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.14"

and run it with a simple...

fn main():
  print("hello, magic!")

This time, the project builds, but I notice that lightbug_http.nojopkg and small_time.mojopkg files being pulled in, have exactly the same date/time stamps as before, in my failing project.

[Sat Mar 01 2025  7:44pm (GMT+0000)]    
~/Code/Mojo/test_small_time/.magic/envs/default/lib/mojo  ll lightbug_http.mojopkg 
.rw-r--r--@ 3.0M carlcaulkett  1 Mar 13:21 lightbug_http.mojopkg

[Sat Mar 01 2025  7:44pm (GMT+0000)]    
~/Code/Mojo/test_small_time/.magic/envs/default/lib/mojo  ll small_time.mojopkg 
.rw-r--r--@ 1.1M carlcaulkett 26 Dec  2024 small_time.mojopkg

[fair bloom]

This shows that with a new project it's still pulling in the old version of small_time, but the project is building. But in any case, the fact remains that I have no control over which version of small_time is used...

UPDATE: If I add the line...

from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server

to the start of the new test project, it also fails to build, with...

/Users/carlcaulkett/Code/Mojo/test_small_time/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/test_small_time/hello.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/test_small_time/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

Over to you guys, @inland scaffoldand @arctic cloak 😉

[fair bloom]

If I put a different import line, such as...

from os import listdir

The project build just fine. This does seem to suggest that it is something to do with lightbug_http and/or small_time...

[fair bloom]

@inland scaffold Sorry to keep bothering you, but in the latest version of lightbug_http just pulled down from Github, I notice in your mojoproject.toml, you've got...

[dependencies]
max = ">=25.1.0,<26"
small_time = "==25.1.0"

Is this a type?

[inland scaffold]

Actually that's the latest version I saw on the mojo-community channel, hmm maybe I got that wrong? @arctic cloak
https://prefix.dev/channels/mojo-community/packages/small_time

[arctic cloak]

Nope that’s right! I changed the versioning scheme

[fair bloom]

@inland scaffold and @arctic cloak, well, something s not working and , as far as I can see, it's nothing that I have any control over from my end! I'm happy to be told otherwise , of course 😉

[inland scaffold]

It's weird that I can't reproduce this locally, you said you tried this on a completely new, clean project?

[fair bloom]

@inland scaffold With tomlproject.toml with...

[dependencies]
max = "=24.6"
lightbug_http = "=0.1.11"

I get a bunch of Mojo errors but nothing to do with the small_time dependency. These errors were, I think because of trying to catch up with v25.1 of Mojo.

with...

[dependencies]
max = "=24.6"
lightbug_http = "=0.1.12"

I just get...

  × failed to solve the conda requirements of 'default' 'osx-arm64'
  ╰─▶ Cannot solve the request because of: No candidates were found for lightbug_http 0.1.12.*.

because, presumably you didn't publish 0.1.12

it's when I move to...

max = "=24.6"
lightbug_http = "=0.1.13"

that, for the first time, I get all the stuff about unknown attribute code: 36... though nothing about small_time here 🤔

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: unknown attribute code: 36
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
     ^

It's the same with...

[dependencies]
max = "=24.6"
lightbug_http = "=0.1.14"

I am perplexed 🙃

[fair bloom]

Yes! see #1238682851717812245 message. It's only when I added the lightbug_http import clauses that I started to get build errors...

[fair bloom]

Also, it's only after deleting .magic and magic.lock that the stuff about small_time starts appearing again...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[fair bloom]

What I think #1238682851717812245 message shows is that something happened in the move from lightbug_http 0.1.11 to lightbug_http 0.1.13 that introduced the problems with error: unknown attribute code: 36 and error: unexpected trailing bytes after Attribute entry.

[arctic cloak]

@inland scaffold Looks like the recipe needs to be updated: https://github.com/Lightbug-HQ/lightbug_http/blob/main/recipes/recipe.yaml#L22

[fair bloom]

Interesting! As far as I can see, the recipe.toml has referenced max >=24.6.0 and small_time == 0.1.6 since 01.01.2025 up until now (apart from a brief glitch with a hyphen in place of an Underscore roundabout 0.8.02.2025 😉). I've tried to find out exactly what recipe.toml actually does, but kapa ai doesn't have a clue 😮

[fair bloom]

Okay, it seems to be related to the hosting of projects at https://repo.prefix.dev/mojo-community...

[inland scaffold]

That's a great lead actually! Seems like the recipe was off for a while.
Just made another hotfix release, @fair bloom can you try 0.1.15?

[fair bloom]

Okay, lots of errors, but all of them Mojo ones, to do with the Max 25.1.0 upgrade. I can do something about this. Thanks for the heads up 🙏🏽

[inland scaffold]

Thank you @arctic cloak for spotting this 🤝

[fair bloom]

@inland scaffold @arctic cloak I think we're up and running... can you see this http://132.145.47.167:8080/ ? 🤞

[inland scaffold]

Yup , I see it!

[fair bloom]

😃 👍

.
The funny thing is that I went through the steps of my workflow to update everything just now, and went to access the page, and didn't see anything. I did a docker logs {container_id} to see what was occurring, and got this...

[opc@ca-web-arm ~]$ docker logs 47a8f0694d63
/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import *
^
/ca_web/client.mojo:49:20: error: 'PageHandler' has no 'HTTPService' member
struct PageHandler(HTTPService):
                   ^~~~~~~~~~~
/ca_web/client.mojo:179:18: error: use of unknown declaration 'Server'
    var server = Server()
                 ^~~~~~
Included from /ca_web/client.mojo:4:
/ca_web/colors.mojo:10:15: warning: 'inout' syntax deprecated, please use 'mut' instead
  fn __init__(inout self, color_name: String, hex: String, r: UInt8, g: UInt8, b: UInt8):
              ^~~~~
              mut
Included from /ca_web/client.mojo:9:
/ca_web/script.mojo:6:15: warning: 'inout' syntax deprecated, please use 'mut' instead
  fn __init__(inout self, id: String):
              ^~~~~
              mut
Included from /ca_web/client.mojo:6:
/ca_web/googlefonts.mojo:1814:27: warning: the `str` function is deprecated, use the `String` constructor instead
      var alias_name = str(font).replace(" ", "")
                       ~~~^~~~~~
/ca_web/client.mojo:1:1: note: 'str' declared here
from lightbug_http import *
^
/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module
/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import *
^

It was only then that I realised that I'd forgotten to push my changes to Github. D'oh 😉

[fair bloom]

@inland scaffold Interestingly, I was able to change my mojoproject.toml to specify max-nightly, but I cannot move the version of Max beyond 25.1.0. IOW, this works...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max-nightly",
    "https://repo.prefix.dev/mojo-community",
]

description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.2.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.15"

But changing it to max = "*" gives these errors...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: attempting to parse 8 bytes when only 2 remain
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
     ^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

Is this something to do with the recipe.yaml again? Not that I'm bothered at the moment 😉

@inland scaffold I don't think we're quite out of the woods yet. The instance stayed up and running for nearly 40 minutes before something upset it...

CONTAINER ID   IMAGE                               COMMAND               CREATED          STATUS                     PORTS     NAMES
25052cfeaa74   ghcr.io/carlca/magic-image:latest   "magic run default"   44 minutes ago   Exited (1) 5 minutes ago             ca-web-container
f2944de77a11   hello-world                         "/hello"              2 weeks ago      Exited (0) 2 weeks ago               objective_yonath
[opc@ca-web-arm ~]$ docker start ca-web-container
ca-web-container
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED          STATUS                   PORTS                                       NAMES
25052cfeaa74   ghcr.io/carlca/magic-image:latest   "magic run default"   45 minutes ago   Up 3 seconds             0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              2 weeks ago      Exited (0) 2 weeks ago                                               objective_yonath
[opc@ca-web-arm ~]$ docker logs 25052cfeaa74


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to read request
ERROR - ReceiveError: An error occurred while attempting to receive data from the socket. Error code: 104
ERROR - Socket.receive: Failed to read data from connection.
ERROR - TCPConnection.read: Failed to read data from connection.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
[opc@ca-web-arm ~]$

It's a lot more solid than it was before, that's for sure!

[fair bloom]

@inland scaffold Having said that, it's up and running at the moment and has been for over an hour 🤞🙏🏽

[inland scaffold]

Hmm, we might need to add proper error handling in this part. The server shouldn't die because of this

[fair bloom]

It was up for 5 hours, then something made it crash. Nothing to do with me; I was asleep at the time 😉...

Last login: Sun Mar  2 22:22:13 2025 from 86.187.161.177
[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED        STATUS                   PORTS     NAMES
25052cfeaa74   ghcr.io/carlca/magic-image:latest   "magic run default"   14 hours ago   Exited (1) 9 hours ago             ca-web-container
f2944de77a11   hello-world                         "/hello"              2 weeks ago    Exited (0) 2 weeks ago             objective_yonath
[opc@ca-web-arm ~]$ docker logs 25052cfeaa74


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to read request
ERROR - ReceiveError: An error occurred while attempting to receive data from the socket. Error code: 104
ERROR - Socket.receive: Failed to read data from connection.
ERROR - TCPConnection.read: Failed to read data from connection.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1


🔥🐝 Lightbug is listening on http://0.0.0.0:8080
Ready to accept connections...
Unhandled exception caught during execution: Server.serve_connection: Failed to read request
ERROR - ReceiveError: An error occurred while attempting to receive data from the socket. Error code: 104
ERROR - Socket.receive: Failed to read data from connection.
ERROR - TCPConnection.read: Failed to read data from connection.
/ca_web/.magic/envs/default/bin/mojo: error: execution exited with a non-zero result: 1
[opc@ca-web-arm ~]$

[inland scaffold]

yeah, looks like the same error. i can check it out this week

[fair bloom]

Sorry to be the bearer of bad news 😉 As I say, though, it's a lot better than it was 😃 As and when you've got time...

[inland scaffold]

No-no, thanks for all the feedback! It's really valuable , you're catching issues that our integration and unit tests aren't covering 😅

[inland scaffold]

Couldn't reproduce this exact error but pushed a fix that might solve it. Can you try 0.1.16? https://github.com/Lightbug-HQ/lightbug_http/releases/tag/v0.1.16

[fair bloom]

Cool! I'm knee deep in Scala debugging at the moment. I may have to get on to it tomorrow 😉

[fair bloom]

@inland scaffold I've just tried with...

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.16"

and on magic update, I'm getting...

~/Code/Mojo/ca_web  magic update

  × failed to solve the conda requirements of 'default' 'osx-arm64'
  ╰─▶ Cannot solve the request because of: No candidates were found for lightbug_http 0.1.16.*.

Any ideas?

[inland scaffold]

my bad, didn't push the latest version to Prefix. What about now?

[fair bloom]

The instance is up and runing in Oracle Cloud. If I ssh to it, then I can run curl http://localhost:8080 and see the Mojo generated HTML/CSS source. I just cannot connect from my web browser. I think it's an Oracle Cloud issue, and not your library. I'll try again a little bit later 😉

• UPDATE * Strike what I just said! It's up and running. I guess the real test will be how long it stays up and running. As a great philosopher once said, "I'll be back!" 😉

[inland scaffold]

Yes, let's see 😈

[inland scaffold]

I've also got an instance running at https://lightbug.buzz/ , let's see for how long this keeps running

[fair bloom]

Looking good 😃

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED       STATUS                   PORTS                                       NAMES
73ea2da5e470   ghcr.io/carlca/magic-image:latest   "magic run default"   6 hours ago   Up 6 hours               0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              3 weeks ago   Exited (0) 3 weeks ago                                               objective_yonath

[inland scaffold]

Looks good so far on my end as well!

Thanks again for catching the bug

[fair bloom]

Out of interest, what did you change for this?

[inland scaffold]

The main server loop logic, we were raising in some cases and the server would stop running, while we should have just logged/returned a BadRequest to the client, e.g in case if malformated requests.
There are still some improvements to be made there, but I'm taking it one step at a time

[fair bloom]

Are you running on Oracle as well? I've just checked mine and the Docker container is still running but I cannot connect to it any more. Maybe there s a connectivity problem with Oracle... 🤔

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED        STATUS                   PORTS                                       NAMES
73ea2da5e470   ghcr.io/carlca/magic-image:latest   "magic run default"   10 hours ago   Up 10 hours              0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              3 weeks ago    Exited (0) 3 weeks ago                                               objective_yonath
[opc@ca-web-arm ~]$

False alarm! It's still up and running. Here's a virtual pint 🍺

[inland scaffold]

I'm in DigitalOcean, but on a small droplet using swap files to overcome the ram limitations, I guess that's why it's so slow

[rich wigeon]

That's why I recommend Oracle.

Absurd amounts of memory for the price.

[fair bloom]

The price being $0 or £0 or ¢0, of course 😍

[rich wigeon]

Yes

[fair bloom]

Woohoo 🥳

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED        STATUS                   PORTS                                       NAMES
73ea2da5e470   ghcr.io/carlca/magic-image:latest   "magic run default"   22 hours ago   Up 22 hours              0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              3 weeks ago    Exited (0) 3 weeks ago                                               objective_yonath
[opc@ca-web-arm ~]$

[inland scaffold]

Nice 👍

[fair bloom]

However, I think there is a slight problem. Okay, the docker ps -a shows that the instance has been running for 24 hours now! But I had to run docker start ca-web-container on two occasions now in order for me to be able to access the site at http://132.145.47.167:8080/.

[inland scaffold]

Which command did you use to start it initially?

[fair bloom]

I use this convoluted script via ssh to pull in the latest docker image, build it, and restart it...

#!/bin/bash

# Script to update the ca-web Docker container on Oracle Cloud VM

CONTAINER_NAME="ca-web-container"
IMAGE_NAME="ghcr.io/carlca/magic-image:latest" # Replace with your actual GHCR image name if different

echo "--- Starting Docker Container Update Script ---"
echo "Container Name: $CONTAINER_NAME"
echo "Image Name: $IMAGE_NAME"
echo ""

# 1. Stop the existing container (if running)
echo "- Stopping existing container '$CONTAINER_NAME' (if running)..."
docker stop "$CONTAINER_NAME"
STOP_EXIT_CODE=$? # Capture exit code of 'docker stop'

if [ $STOP_EXIT_CODE -eq 0 ]; then
  echo "  ✓ Container '$CONTAINER_NAME' stopped successfully (if it was running)."
else
  echo "  ⓘ Container '$CONTAINER_NAME' was not running or stop command failed (exit code: $STOP_EXIT_CODE). Proceeding with removal."
fi
echo ""

# 2. Remove the existing container (if it exists)
echo "- Removing existing container '$CONTAINER_NAME' (if it exists)..."
docker rm "$CONTAINER_NAME"
RM_EXIT_CODE=$? # Capture exit code of 'docker rm'

if [ $RM_EXIT_CODE -eq 0 ]; then
  echo "  ✓ Container '$CONTAINER_NAME' removed successfully (if it existed)."
else
  echo "  ⓘ Container '$CONTAINER_NAME' did not exist or remove command failed (exit code: $RM_EXIT_CODE). Proceeding with image pull."
fi
echo ""

# 3. Pull the latest Docker image from GHCR
echo "- Pulling latest Docker image '$IMAGE_NAME' from GHCR..."
docker pull "$IMAGE_NAME"
PULL_EXIT_CODE=$? # Capture exit code of 'docker pull'

if [ $PULL_EXIT_CODE -eq 0 ]; then
  echo "  ✓ Docker image '$IMAGE_NAME' pulled successfully."
else
  echo "  ✗ ERROR: Failed to pull Docker image '$IMAGE_NAME' (exit code: $PULL_EXIT_CODE). Aborting."
  exit 1 # Exit script with error code
fi
echo ""

# 4. Run a new container from the pulled image
echo "- Running new container '$CONTAINER_NAME' from image '$IMAGE_NAME'..."
docker run -d -p 8080:8080 --name "$CONTAINER_NAME" "$IMAGE_NAME"
RUN_EXIT_CODE=$? # Capture exit code of 'docker run'

if [ $RUN_EXIT_CODE -eq 0 ]; then
  echo "  ✓ New container '$CONTAINER_NAME' started successfully."
else
  echo "  ✗ ERROR: Failed to run new container '$CONTAINER_NAME' (exit code: $RUN_EXIT_CODE). Please check for errors above."
  exit 1 # Exit script with error code
fi
echo ""

echo "--- Docker Container Update Script Completed ---"
echo "Application should now be updated and running in container '$CONTAINER_NAME'."
echo "Access it at http://<your_instance_public_ip_address>:8080" # Remind user to use correct URL

docker start "$CONTAINER_NAME"

exit 0 # Exit script with success code

[inland scaffold]

@fair bloom asked Claude, it suggests a couple changes in these parts of the script (only sent it the latest section):

#!/bin/bash

# Set variables
CONTAINER_NAME="your-container-name"
IMAGE_NAME="ghcr.io/your-org/your-image:latest"

echo "--- Docker Container Update Script Starting ---"

# 1. Check if Docker is running
echo "- Checking if Docker is running..."
if ! docker info > /dev/null 2>&1; then
  echo "  ✗ ERROR: Docker is not running. Please start Docker and try again."
  exit 1
fi
echo "  ✓ Docker is running."
echo ""

# 2. Stop and remove the existing container (if it exists)
echo "- Stopping and removing existing container '$CONTAINER_NAME' (if it exists)..."
docker stop "$CONTAINER_NAME" > /dev/null 2>&1
docker rm "$CONTAINER_NAME" > /dev/null 2>&1
echo "  ✓ Container '$CONTAINER_NAME' stopped and removed if it existed."
echo ""

# 3. Pull the latest Docker image from GHCR
echo "- Pulling latest Docker image '$IMAGE_NAME' from GHCR..."
docker pull "$IMAGE_NAME"
PULL_EXIT_CODE=$? # Capture exit code of 'docker pull'

if [ $PULL_EXIT_CODE -eq 0 ]; then
  echo "  ✓ Docker image '$IMAGE_NAME' pulled successfully."
else
  echo "  ✗ ERROR: Failed to pull Docker image '$IMAGE_NAME' (exit code: $PULL_EXIT_CODE). Aborting."
  exit 1 # Exit script with error code
fi
echo ""

# 4. Run a new container from the pulled image with restart policy
echo "- Running new container '$CONTAINER_NAME' from image '$IMAGE_NAME'..."
docker run -d \
  -p 8080:8080 \
  --name "$CONTAINER_NAME" \
  --restart unless-stopped \
  "$IMAGE_NAME"

RUN_EXIT_CODE=$? # Capture exit code of 'docker run'

if [ $RUN_EXIT_CODE -eq 0 ]; then
  echo "  ✓ New container '$CONTAINER_NAME' started successfully."
else
  echo "  ✗ ERROR: Failed to run new container '$CONTAINER_NAME' (exit code: $RUN_EXIT_CODE). Please check for errors above."
  exit 1 # Exit script with error code
fi
echo ""

# 5. Verify the container is running
echo "- Verifying container is running..."
if docker ps | grep -q "$CONTAINER_NAME"; then
  echo "  ✓ Container '$CONTAINER_NAME' is running."
else
  echo "  ✗ WARNING: Container '$CONTAINER_NAME' is not running. Check the logs with 'docker logs $CONTAINER_NAME'."
fi
echo ""

echo "--- Docker Container Update Script Completed ---"
echo "Application should now be updated and running in container '$CONTAINER_NAME'."
echo "Access it at http://<your_instance_public_ip_address>:8080"

exit 0 # Exit script with success code

1. You're using docker run -d to run the container in detached mode, which is correct for running in the background.

2. At the end of your script, you're executing docker start "$CONTAINER_NAME" unnecessarily since the container should already be running from the docker run command.
3. Most importantly, your container is likely stopping immediately after starting because there's no process keeping it alive.

The Solution

To keep your container running without needing to manually restart it, you need to ensure:

1. The container's main process stays running: Docker containers stop when their main process exits.
2. Remove the redundant docker start command: It's not needed if your container is configured correctly.

[fair bloom]

Interesting! Looks like we have a Gemini 2.0 Flash vs Claude AI faceoff 😉

I agree about the redundent docker start command. Also, I've discovered that after running this script, it seems to take Oracle about a minute or so before it will star receiving web requests again.

I like the addition of the --restart unless-stopped flag. I've just looked it up and it seems totally appropriate for my situation.

Thanks for the heads up 🙏🏽

[fair bloom]

</html>[opc@ca-web-adocker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED        STATUS                   PORTS                                       NAMES
c49b65e8f80d   ghcr.io/carlca/magic-image:latest   "magic run default"   26 hours ago   Up 26 hours              0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              3 weeks ago    Exited (0) 3 weeks ago                                               objective_yonath
[opc@ca-web-arm ~]$

@inland scaffold It still has problems, occasionally, connecting to the web site, but I think this may be more to do with the fact that I'm using the free-tier service from Oracle (words that I thought I would never use in the same sentence 😉) rather than any software related issue.

[inland scaffold]

https://lightbug.buzz/ is also buzzing along so far

[astral sundial]

Congrats @inland scaffold, you just advanced to level 10!

[fair bloom]

Still running after 3 days 🎉🥳🍻

[opc@ca-web-arm ~]$ docker ps -a
CONTAINER ID   IMAGE                               COMMAND               CREATED       STATUS                   PORTS                                       NAMES
c49b65e8f80d   ghcr.io/carlca/magic-image:latest   "magic run default"   3 days ago    Up 3 days                0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   ca-web-container
f2944de77a11   hello-world                         "/hello"              4 weeks ago   Exited (0) 4 weeks ago                                               objective_yonath
[opc@ca-web-arm ~]$

[inland scaffold]

I guess we can call Lightbug production-ready now 😁 (joking)

[stray shadow]

Great! 🥳

[fair bloom]

@inland scaffold I do't know if it's because of the new Mojo build, but my ca_web repo is suddenly not building, with...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: unexpected trailing bytes after Attribute entry
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
     ^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

Has anything changed in your side?
Any chance of having a look? https://github.com/carlca/ca_web.git

[astral sundial]

Congrats @fair bloom, you just advanced to level 17!

[arctic cloak]

Ohh the recipe file was probably missed again, I’ll open a pr to update it later today if that was the problem

Part of the build and deploy process for the package into the conda channel

[fair bloom]

Yeah! I had a sneaking suspicion that it was somegthing similar to last time - the error messages didn't seem like standard programming error messages...

[arctic cloak]

It does look like @inland scaffold already updated the recipe 🤔 can you show the versions of lightbug_http and max you’re using?

[fair bloom]

Here is my mojoproject.toml. Nothing has changed since halfway through March when we were celebrating many days of continuous live deployment at Oracle...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max-nightly",
    "https://repo.prefix.dev/mojo-community",
]

description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.2.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "=25.1.0"
lightbug_http = ">=0.1.16"

[arctic cloak]

try pinning to "=0.1.16" instead?

[fair bloom]

Great! That worked 🙂

[arctic cloak]

Great! 👍

I think its best to pin to a specific version, since new versions usually come out with a new max version. So they'll never be backwards compatible (until Mojo hits a level of stability)

[fair bloom]

Does this mean that Lightbug has moved to a 0.1.17 version which is not compatible?

[arctic cloak]

Yep, Max 25.2.0 and Lightbug_http 0.1.17

[fair bloom]

I tried chaging to...

[dependencies]         
max = "=25.2.0"
lightbug_http = "=0.1.17"

but that failed with...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unknown attribute code: 36
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

Any ideas?

Looks like it's back to...

[dependencies]         
max = "=25.1.0"
lightbug_http = "=0.1.16"

For the time being. Thoughts, @inland scaffold?

[arctic cloak]

Can you also add this channel in the list of channels?

"https://repo.prefix.dev/modular-community"

small_time is now being sourced from the modular-community

[fair bloom]

I tried it with...

[project]
authors = ["Carl Caulkett <[email protected]>"]
channels = [
    "conda-forge",
    "https://conda.modular.com/max-nightly",
    "https://repo.prefix.dev/mojo-community",
    "https://repo.prefix.dev/modular-community",
]

description = "Testing a2svior's lightbug_http web server package"
name = "ca_web"
platforms = ["osx-arm64"]
version = "0.2.0"

[tasks]
default = "magic run mojo client.mojo"

[dependencies]
max = "=25.2.0"
lightbug_http = "=0.1.17"

but i get the same error...

/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/lib/mojo/small_time.mojopkg:0:0: error: unknown attribute code: 36
/Users/carlcaulkett/Code/Mojo/ca_web/client.mojo:1:1: error: failed to materialize top-level module
from lightbug_http import HTTPService, HTTPRequest, HTTPResponse, OK, NotFound, Server
^
/Users/carlcaulkett/Code/Mojo/ca_web/.magic/envs/default/bin/mojo: error: failed to parse the provided Mojo source module

[fair bloom]

In any case it's all a bit immaterial because those scumbags at Oracle have just deleted the free tier account that I had foolishly and naively thought would continue indefinitely. @rich wigeon, do you know why Oracle would just delete my account with no warning or opportunity to extend it?

[rich wigeon]

No idea

I've never had that happen.

I've seen it happen to a few people, so it's possible they're cleaning up free tier only users.

[fair bloom]

This was the email I received a few days ago...

Dear Customer,

This letter is to inform you that your Cloud Services described below were terminated on Monday, April 7, 2025 03:16 AM Coordinated Universal Time.

The termination stopped all running services and purged the data. All data was deleted and is non-recoverable.

For any further assistance, please contact Oracle Support.

Absolutely no warning, whatsoever 😡 As a piece of customer relations, it stinks to high heaven, and will ensure that I never have anything to do with Oracle again.

[rich wigeon]

Wow, that's aggressive.

They must be losing money.

[fair bloom]

I've just been reading about Larry Ellison on Wikipedia; I don't think he's going to starve any time soon 😡

[inland scaffold]

Actually 0.1.17 wasn't published to modular-community yet, but hopefully it will be soon!
Re: this looks very fishy, were there at least any warnings before they decided to wipe all of your data? Crazy

[fair bloom]

No, there was no warning at all! I tweeted Oracle Cloud to point out the issue, and then I tweeted Larry Ellison himself. I doubt whether he will respond...

[delicate scroll]

Trying to run example code,
This fails:
from lightbug_http import HTTPRequest

/Users/alex/IdeaProjects/mojostuff/.pixi/envs/default/lib/mojo/lightbug_http.mojopkg:0:0: error: expected M::KGEN::ParamDeclArrayAttr, but got: !kgen.generator<!lit.generator<<trait<@stdlib::@utils::@write::@Writer>>[2](!lit.ref<@stdlib::@builtin::@string_literal::@StringLiteral<:string " NEWREF PyBool_FromLong, refcnt:">, imm *[0,0]> read_mem, !lit.ref<:trait<@stdlib::@utils::@write::@Writer> *(0,0), mut *[0,1]> mut, |) -> !kgen.none>>
/Users/alex/IdeaProjects/mojostuff/hello.mojo:1:6: error: failed to materialize top-level module
from lightbug_http import HTTPRequest
     ^
mojo: error: failed to parse the provided Mojo source module

I tried this:
from lightbug_http.http import HTTPRequest

It also fails:

/Users/alex/IdeaProjects/mojostuff/hello.mojo:1:20: error:
from lightbug_http.http import HTTPRequest
                   ^
mojo: error: failed to parse the provided Mojo source module

@inland scaffold is the problem on my end on somewhere else?

[inland scaffold]

Hi! May I ask which MAX/Lightbug version you're on (maybe just paste mojoproject.toml) and the OS?

Will try to reproduce

[delicate scroll]

@inland scaffold

[workspace]
authors = ["Alex"]
channels = ["https://conda.modular.com/max-nightly", "conda-forge", "https://repo.prefix.dev/modular-community"]
name = "mojostuff"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
modular = ">=25.5.0.dev2025070405,<26"
lightbug_http = ">=25.4.0"

MacOS Sonoma 14.3.1

[arctic cloak]

@delicate scroll There’s a small issue with the modular package build process that marks the upper constraint bound as too permissive. Lightbug 25.4.0 will not work with the 25.5.0 modular nightlies, so you should switch to 25.4.0

[delicate scroll]

@arctic cloak thank you, it solved the problem