Mongoose - you may not know it yet but your game really needs an embedded Http server!

Mongoose a tiny embeddable Http server made from only a single .h and .c  file. How awesome is that?


Very awesome! And it leads me to something I've wanted to implement for a while now. I like Lua as a programming language it's great, it's basically Scheme with a C-style interface, it's really easy to embed and is a on it's way to becoming the lingua franca scripting language of the game industry.

For my projects I usually embed Lua in some OpenGL graphics engine. The game runs the Lua scripts and displays the nice graphics. I edit Lua scripts using Sublime Text 2, a really slick text editor. (I hope that screencast shows Ctrl-D and the alt-shift-downkey shortcuts because they are Sublime Text's winning feature for me).

When I make my projects I usually use a little trick to make development go a little more smoothly. If you press a certain key - I use F2 - the project resets it's internal state and reloads it's Lua script. So you don't need to restart the program - no compile times or anything like that. You just edit the script, press F2 and almost instantaneously your new code is running. This is a feature I really missed from Love2d when I played with it (so if your listening Love2d contributors please add this feature :)).

But that's not good enough! We can do better!


The current development process goes like this:

1. Boot project
2. Move focus to sublime text
3. Edit Lua
4. Move focus to project
5. Press F2
6. If there's an error go back to sublime text
7. Play game and see what needs to be done next

And this brings me to how I'd like to improve this work flow. I want to be able to reload my script files in my project from Sublime Text. On one screen I could be hacking away and on the other screen have my project running. I hit F2 and glance at the project screen - it says there's an error - so I find it, fix it up and continue hacking away. Much more streamline! The project could even be running on a totally separate computer!


I've been toying with this for a bit, initially I was using sockets and also looking a enet but I wanted something cross-platform, more highlevel and easy to use from the Sublime-side. This is where Mongoose comes in - I can simply add a small HttpServer to my project. Sublime Text 2 uses python as an extension language and it's easy to send Http requests from python, including postdata.

Once this basic system is in place it could be extended in a number of interesting ways:

• Send over selected code to be updated on the fly. Like single functions or globals.
• If there's an error have the project send the line number back to sublime and have it jump there.
• Have a webpage where I can see the _G table
• Have some ajax where I can modify the values in the _G table

I'm not there yet but it's something I've found myself playing around with. Here's my current version of the Mongoose server. To get it to build under mingw I had to add

#define _strtoi64(a,b,c) strtoll(a,b,c)

to the header file. I'm far more comfortable in an IDE so it always takes me far to long to work out the correct way to build this type of stuff in mingw. Here's the build command I use to g++ to embed it. (There are loads of warnings mainly due to C / C++ differences)

g++ -W -Wall -mthreads -I../lib/mongoose/ -DNDEBUG -Os ../lib/mongoose/mongoose.c httpserver.cpp main.cpp -lws2_32 -ladvapi32 -fpermissive -Wunused-parameter -static-libgcc -static-libstdc++ -o servern.exe

Here's the server code .h file. As usual moving my code to Blogger has upset the indentation :(

#ifndef HTTPSERVER_H
#define HTTPSERVER_H

#include 
#include 
struct mg_context;
struct mg_request_info;
enum mg_event;

struct HttpRequest
{
 std::string mURI;
 std::string mPostData;
};

//
// Creates a HttpServer to listen on a port for incoming requests.
// If there's a request pending PopRequest will return true and fill in the HttpRequest struct.
//
class HttpServer
{
private:
 mg_context* mContext;
 std::queue mMsgQueue;
 static void* callback(enum mg_event event, struct mg_connection *conn, const struct mg_request_info *request_info); 
public:

 bool PopRequest(HttpRequest& request);

 HttpServer(int port);
 ~HttpServer();
};

#endif

Pretty simple and here's the .cpp actual implementation.

#include 
#include 
#include 
#include 
#include "HttpServer.h"
#include 

static void* HttpServer::callback(enum mg_event event,
                      struct mg_connection *conn,
                      const struct mg_request_info *request_info) 
{
 printf("Callback\n");

 if (event == MG_NEW_REQUEST) 
 {
  char content[1024];
  int content_length = snprintf(content, sizeof(content),
                                "Hello from servern! Remote port: %d",
                                request_info->remote_port);

  assert(request_info->user_data);
  HttpServer* server = static_cast(request_info->user_data);
  const char* cl = mg_get_header(conn, "Content-Length");

  if(!strcmp(request_info->request_method, "POST")
   && cl != NULL)
  {
   // handle post data
   size_t buf_len = atoi(cl);
   char* buffer = (char*) malloc(buf_len+1);
   mg_read(conn, buffer, buf_len); 

buffer[buf_len] = '\0';
   HttpRequest request;
   request.mURI = request_info->uri;
   request.mPostData = buffer;
   // Lock stack while a write is done
   server->mMsgQueue.push(request);
   free(buffer);
  }

  mg_printf(conn,
            "HTTP/1.1 200 OK\r\n"
            "Content-Type: text/plain\r\n"
            "Content-Length: %d\r\n"        // Always set Content-Length
            "\r\n"
            "%s",
            content_length, content);
  // Mark as processed
  return (void*)"";
 } 
 else 
 {
  return NULL;
 }
}

bool HttpServer::PopRequest(HttpRequest& request)
{
 if(mMsgQueue.empty())
 {
  return false;
 }

 // Copy data request.
 // 1. Simple  - one pop operation instead of a peek and pop
 //        or passing over memory management
 // 2. This is going to be used for developement with low / throughput. 
 //   If this changes then this should also change.
 request = mMsgQueue.front();
 mMsgQueue.pop();
 return true;
}

HttpServer::HttpServer(int port)
{
 const char *options[] = {"listening_ports", "8080", NULL};
 mContext = mg_start(&HttpServer::callback, this, options);
}


HttpServer::~HttpServer()
{
 mg_stop(mContext);
}

I found the Mongoose docs to be a bit lacking but I think the static callback function maybe being called from a different thread and if that's the case there needs to be a lock around the access to the queue. Everything else should be pretty straightforward I think. It's been a while since I've used C++ so if there's any glaringly wrong here, please let me know!

The use case is very simple - check if it has a message, if it does, then do something with it. There's no RESTful interface stuff here! But it wouldn't be hard to build in.

Here's my main loop.

HttpRequest request;
HttpServer httpServer(8080);

while(true)
{
 if(httpServer.PopRequest(request))
 {
  printf("Request made:%s\n", request.mPostData.c_str());
 }
}

To test the postdata I use a small python program.

import urllib
import httplib2
import sys

if len(sys.argv) != 2:
    print "Please enter message"
    raise SystemExit

msg = sys.argv[1]

username = "user"
password = "pass"

http = httplib2.Http()
#http.add_credentials(username, password)

response = None
try:
 response = http.request(
     "http://127.0.0.1:8080", 
     "POST", 
     urllib.urlencode({"status": msg})
 )
except:
    print "Unexpected error:", sys.exc_info()[0]

I'm not too far away from getting this all together, it's been pretty fun so far and will hopefully make my development process faster. I suffer strongly from the Babbage curse and I'm really feeling some pressure to release a game instead of constantly working on little bits of random projects! :D

It seems Sublime Text 2 doesn't use the standard installed Python, instead it has it's own local copy of python 2.6. This python doesn't have httplib2 but it does have httplib which is near enough. Creating a plugin is pretty simple in Sublime Text - just create a .py file in your user folder:

C:\Users\MyUsername\AppData\Roaming\Sublime Text 2\Packages\User

Whatever you call that .py file will be how you refer to it later, to call it. I called this one "reload" for now. Here's the plugin code. I've hacked this together quickly it could be prettier.

import sublime
import sublime_plugin
import urllib
import httplib
import sys

print('plugin loaded')


class Reload(sublime_plugin.TextCommand):
 def run(self, edit):
  try:
   msg = "reload"
   host = "127.0.0.1:8080"
   http = httplib.HTTPConnection(host)
   # additional path goes below like /reload/
   http.putrequest("POST", "/")
   http.putheader("content-length", str(len(msg)))
   http.endheaders()

   http.send(msg)
   response = http.getresponse()
   print self.view.file_name(), "Reload command sent."
  except:
   print self.view.file_name(), "Reload failed. %s" % sys.exc_info()[0]

In the Sublime menu select View→Show Console. Then in the console you can type "view.run_command('reload')" and that will run it. But a key shortcut would be better. In the menu select Preferences→Key Bindings (User) and add the following shortcut:

[
 { "keys": ["f2"], "command": "reload" }
]

Now when you press F2 you'll send a message to your server!