Context

I’ve tried a few times to learn Japanese. All of those attempts have ended with me quitting after a few months and feeling pretty bad at it. This year I began a final attempt after I visited Japan and found new motivation. This attempt (the final attempt) began on 3/9/2023 starting from nearly zero (I could remember most of the kana and a few set phrases). This is the first yearly update which documents my experience learning the language this year.

2023 Milestones

• Completed Genki 1 on 8/18/2023 (took 170 days)
• Completed Tango N5 on 9/4/2023 (took 78 days)
• Completed Genki 2 on 12/30/2023 (took 109 days)
• Learned 1970 new words

7/5/23 entry

Kanji
I began 3/9/2023. I reset my WaniKani level back to 0 from 10, which hurt. It felt like I was finally admitting to myself that I had failed in my last attempt. Level 10 wasn’t even that high, but it was still several hundred kanji that I spent many many hours learning. After I got going again, I realized that I hadn’t really failed, it was only a setback. I may not have remembered every kanji, but I was certainly learning them much faster than last time. As of writing I’m back up to level 8 already.

Apparently this covers 94% of the JLPT numbers, but I’m still having trouble seeing new kanji in the Tango N5 deck I use. Perhaps this is because there seems to be some sort of disconnect between kanji JLPT levels and the JLPT levels for words that use those kanji. For example, the Tango N5 deck contains the word 辞書, “dictionary”. I haven’t seen either of those kanji, because 辞 is an N3 level kanji but the entire word is considered N5 level. I don’t fully understand why this disconnect exists yet, maybe I’ll have a better understanding in the future.

Either way, I’m actually attempting to slow down my kanji learning speed. Previously I’ve been doing 10 new lessons per day (and all of the radicals at once). This has been great for learning kanji, but it takes up a lot of time and I feel now that I’m falling behind on grammar. I’ll be moving to 5 new lessons per day in order to give myself more time for Genki.

毎日、漢字を覚えます！

8/18/23 entry

Grammar
I started with Genki 1 pretty much as soon as I got back from Japan, 3/1/23. I was able to recognize all of the kana after a day or two of re-learning. I signed up for the Seattle Japanese Language school classes, which ran 4/4/23 - 6/6/23. These class covered Genki 1 lessons 3 and 4. During this time, I felt like this progression was too slow for me. I enjoyed getting a chance to speak in class and I think this format probably works well for a lot of people, but I would have preferred a faster format. These classes were primarily useful for making me feel accountable for learning and helping me to form some discipline. I did not study ahead during this time.

After these classes ended, I began working through Genki 1 on my own and at a much faster pace. I completed Genki 1 on 8/18/23, meaning I spent roughly 5.5 months on it. I did the exercises in both the textbook and workbook. Towards the end I felt that writing was slowing me down too much, so I transitioned to using https://sethclydesdale.github.io/genki-study-resources to answer the questions. I did not practice the kanji or study the vocabulary from the textbook, so when I encountered a word I did not recognize I just looked it up in a dictionary.

I noticed towards the end of Genki 1 that exercises would involve lots of grammar points and I wouldn’t always be able to recall them, so I had to look through my notes. Now that I’ve finished working through all of the exercises, I’m planning on adding the grammar points to Bunpro so that I can etch these into my brain via SRS. I tried doing this as I went through Genki 1 but the exercises would use grammar points that I had not encountered yet. I found the casual (short-form) stuff particularly frustrating because I hadn’t gotten to that part of the book yet.

I plan on picking up Genki 2 soon and I’m hoping to finish at around the end of this year.

Vocabulary
I started by using an Anki deck to SRS the Genki 1 vocabulary, but I found that to be frustratingly slow and I think a lot of the words weren’t very useful, so I stopped doing that. On 6/19/23 I started the Tango N5 Anki deck and as of today I’ve got about 270 new cards left. I’m working at a pace of 15 new cards/day which feels about as fast as I can go. I feel like this is significantly more enjoyable than learning vocabulary on WaniKani or through the Genki 1 Anki deck. I feel like the sentence context helps to read the unknown kanji and this process has made me question whether or not it even makes sense to learn kanji in isolation, more on that in the kanji section.

I’m going to continue through the Tango N5 deck and then tackle Tango N4. I have a copy of よつばと！ sitting on a shelf next to me that I’d love to start reading. I don’t think 1000 words is going to cut it though, so I may end up using the Yotsubato! reading pack (https://livingjapanese.com/pages/yotsubato_reading_pack1.html) to help me out. I think I’ll try this in tandem with Tango N4.

I really want to get started consuming native materials ASAP. Lots of folks online recommend anime, but I’m not the biggest fan of it. I’ll probably try it out to see if it helps me learn. I do really like Ghibli movies though, so I want to try to learn enough vocab to watch Totoro. I found jpdb.io which has has a vocabulary deck for the movie that looks promising.

Kanji
I haven’t made much more progress on WaniKani. I do my reviews every day but this is easily the least enjoyable part of my learning. I feel like learning kanji in isolation isn’t helping me nearly as much as just learning the vocabulary directly. I’m going to keep doing my reviews, but I think I’ll put a hold on new lessons for a bit. I don’t want to fall behind, but this feels pretty inefficient and doesn’t motivate me very much.

Listening/Speaking
🙈

9/4/23 entry

Vocabulary
Today I finished the Tango N5 deck! Or, at least I’ve gone through all the new cards. It ended up taking me 78 days. I averaged 25 mins/day and it took about 33 hours in total. I answered a little over 10,000 questions. I feel great having completed it and I’m excited to move on to the N4 stuff! That’s +1171 vocab and with my Wanikani progress I estimate I’m around 1500 vocab. I took a look at Satori Reader yesterday and despite learning soooo many new words every day, I can hardly recognize any words in the beginner stories.

End-of-year entry

Grammar
I’m wrapping up Genki 2 right now. I’ve got 2 or 3 remaining grammar points to work on and then I’ll have finished Genki! For Genki 2, I read through the grammar point explanations in the book, but instead of working through the book exercises I added the grammar points to my Bunpro SRS reviews and have worked on the reviews for those daily. Each day I’ve added 1 or 2 grammar points, with and exception for a month or two where I did not have much time for learning new material.

As I’ve added more grammar points, I’ve noticed that the examples in Genki have become more difficult as there are more past grammar points that they build on. I think the Bunpro reviews are helping me to retain a lot of this, but I feel like I haven’t really “nailed down” a lot of due to the lack of reading. Since I’m just doing a bunch of SRS exercises, I get exposed to a lot of quizzes but not actual usage of the grammar points which is making me feel like it is difficult to read longer sentences.

Vocabulary
According to Anki, I’ve got about 158 new words left in the Tango N4 deck. As of this writing (12/24/23) I’ve put 1930 words into young/mature. As I started Tango N5 on June 19th that comes out to a little over 10 new words per day. My pace for the N4 deck has been slower at 10 words per day, and I missed about a month of new cards due to time constraints, but I’ve kept up my reviews. I’m looking forward to wrapping up the new cards in this deck so that I can move on to creating my own cards through immersion. The Tango decks are marketed as i+1 on forums online, but I have found that the ordering of these cards is often super weird. I’m seeing cards for words at the end of N4 that showed up midway through N5. I’m also finding that the whole-sentence nature of these cards makes it so that reviews take longer as I’m reading through the entire card and checking against multiple words.

Again I’m worried that the sentence nature of the Tango deck is causing me to remember positions of words rather than the words themselves. I’m hoping that I’m actually learning something here though, and I think that will be put to the test when I move onto word-only cards in January.

Kanji
This is tricky, I haven’t been working on isolated kanji since I stopped WaniKani back in August. I’m not sure if I should pick this back up, as I have been noticing that it is becoming more difficult to differentiate the kanji I’m seeing in words as I add on more known words. This is especially tricky when reading sentences that aren’t part of the Tango decks as I don’t have the “muscle memory” for them.

I’m thinking of picking up isolated kanji learning again to help with this. Perhaps using the Migaku kanji god extension to build kanji cards from my vocabulary cards will make this process easier? I just don’t like using prebuilt decks very much and I don’t think I can make myself do much more WaniKani.

Listening / Speaking
I’ve still not put much practice into this yet. I feel like I’m too far behind everywhere else to get much out of this. Though I have explored using SuperNative.tv a bit. Currently my MMR on the Listen + Recall section is 1614. This seems pretty fun and I think would be useful for boosting my listening skills. I’m a bit lost on figuring out speaking skills without doing an actual conversion though, which sounds very difficult right now.

Immersion
I’ve made some attempts through both Yotsuba and Satori Reader. I found Yotsubato to be more difficult due to the colloquial language preventing me from getting much use out of Yomichan. Additionally I have come to rely on the kanji to help with many words due to the Tango decks, so having to figure out the words from kana alone is difficult. I haven’t made it through more than a few dozen pages and I want to put more effort into this next year (see below goals).

Satori Reader has been much easier. I have read through about 20 episodes now, mostly through 壁の穴. I think the episodes are pretty good on the difficulty, and I have noticed that overwhelmingly most of my troubles come from not knowing the grammar. Having to look up a few words per episode is fine, but I’m noticing there’s a huge rift between knowing how to answer the grammar SRS on Bunpro and being able to actually read real sentences that use the grammar.

Overall immersion has been kind of exhausting. I feel like it will continue to be for a little while until I’ve read enough though, but I plan on powering through it next year and I’m hoping that it will become enjoyable before long. Additionally, immersion will very soon become my main source of vocabulary as I’ll be mining for all future words starting in just a few days.

Goals for 2024

Theme: reach N3 level and then some.

• Learn 1825 new words
  • At 5 words per day, this would bring my total known word count to nearly 4000, which is well beyond the 3000 reported words needed for N3.
• Complete Quartet 1 and mature the grammar points in Bunpro
  • This should fulfill the grammar component for N3.
• Read 1000 pages of manga in Japanese
  • Assuming I’ll read about about 10 pages/hour. This is 100 hours.
• Complete 2 tutoring sessions
  • This is for practicing speaking and listening.

CENTAURI HEAVY INDUSTRIES

God has crafted you from blood and motor. Your sole purpose is to improve Their mining operation on some long-forgotten planet. You’ve got 30 minutes till your body crumbles.

About

CENTAURI HEAVY INDUSTRIES is a narrative-driven resource gathering factory simulation game with multiple endings set in the grim darkness of the far future.

Recently I’ve been reading through Computer Graphics from Scratch. The sections on raytracing have been a great refresher for me, and I’ve learned some new tricks.

I’ve written up a C++ software raytracer based on the book and now I’ve got an animation to show for it! The above video combines about 2200 frames that were rendered as PPM images and stitched together using ffmpeg. Each 900x900 frame takes about 1 second to render on my M1 mac, and the renderer streams the images to an SDL2 window as they finish so that I can see the progress. Lowering the quality of the render allows us to get to real-time, but I prefer the look of the reflections.

When last we left off, we developed a full-featured CHIP-8 emulator. Unfortunately, it is really slow. Why is it slow? Well if you look back at the main loop you’ll see that we’re rendering to the screen after every cycle is executed. With vsync enabled, SDL will try to lock rendering to your monitor’s refresh rate (probably 60Hz). What that means is that the SDLViewer::Update method is going to block for a long time, almost every time it is called, waiting for the monitor vsync to arrive.

So how fast should our emulator execute? Well, that’s tricky to determine exactly. Each opcode has a different execution time in a real computer, though approximate timings are known. We could execute an instruction, lookup how long it should have taken on real hardware, and sleep until we’re ready to go again. The problem with that is that we don’t have access to CPU timings at that resolution. Most of these instructions should take a couple of microseconds, but we can only resonably sleep for at least a millisecond on modern systems.

We can, however, amortize it. We know that over the course of a second, an average of 540 operations should execute. This will of course not be the case if every instruction is something complex like a draw, but it will work for real-world programs. We also know that CHIP-8 machines target 60Hz refresh rates. That means that every 540 / 60 = 9 instructions our emulated machine ought to wait until the next vsync.

So we ought to be able to alter our own main loop to execute 9 instructions before every call to SDLViewer::Update. We’d then be exploiting our own diplay’s refresh rate to correct our emulation timings. Let’s give it a shot:

// main.cpp

void Run() {
  ...
  while (!quit) {
    for (int i = 0; i < 9; i++) {
      cpu.RunCycle();
    }
    cpu.GetFrame()->CopyToRGB24(rgb24, /*r=*/255, /*g=*/0, /*b=*/0);
    viewer.SetFrameRGB24(rgb24, height);
    auto events = viewer.Update();
    ...
  }
}

With that, there’s a pretty good chance that your emulator is now executing 540 operations per second, and is displaying graphics at 60Hz!

There is an issue if you try to run this on a machine using a display with a refresh rate of more than 60Hz, our utilization of our own hardware’s refresh rate is no longer accurate to the emulated machine, causing the emulation to run too fast. If this is an issue for you, you can emulate the vsync as well. You know that the 9 intructions + the render needs to take 1000/60 = 16.67 ms, so you can execute those operations, measure how long they take, and sleep as long as necessary. Because that sleep won’t be the most accurate, you can measure the execution time of 540 operations (60 emulated refresh cycles) as well, and sleep up to the second boundary in order to provide any correction to the vsync sleeps. This is exactly what I do in the original version of this emulator that I based these articles on. That project also uses a seperate thread for CPU execution, whish is likely unnessary (but cool).

Now that we’re executing at 540 instructions per second, supporting the CPU timers is pretty easy. The CHIP-8 features two timers, a delay timer and an audio timer. Both of these timers decrement at 60Hz, and the machine beeps while the audio timer is not zero. We can now simply decrement these timers once every 9 instructions in order to emulate a 60Hz clock:

// cpu_chip8.cpp

void CpuChip8::RunCycle() {
  ... 
  // Update timers
  num_cycles_++;
  if (num_cycles_ % 9 == 0) {
    if (delay_timer_ > 0) delay_timer_--;
    if (sound_timer_ > 0) {
      std::cout << "BEEPING" << std::endl;
      sound_timer_--;
    }
  }
}

One last thing I have yet to mention is input. This is a simple matter of reading the events returned from SDLViewer::Update. Like I mentioned, the CHIP-8 uses a 16-key keypad, and you can map those to whatever keys you’d like on the keyboard. Here’s a snippet:

// main.cpp

...
auto events = viewer.Update();
uint8_t cpu_keypad[16];
for (const auto& e : events) {
  if (e.type == SDL_KEYDOWN || e.type == SDL_KEYUP) {
    if (e.key.keysym.sym == SDLK_1) {                        
      cpu_keypad[0] = e.type == SDL_KEYDOWN;
    } else if (e.key.keysym.sym == SDLK_2) {                        
      cpu_keypad[1] = e.type == SDL_KEYDOWN;
    } else if (e.key.keysym.sym == SDLK_3) {                        
      cpu_keypad[2] = e.type == SDL_KEYDOWN;
    } else if (e.key.keysym.sym == SDLK_4) {                        
      cpu_keypad[3] = e.type == SDL_KEYDOWN;
    } else if (e.key.keysym.sym == SDLK_q) {                        
      cpu_keypad[4] = e.type == SDL_KEYDOWN;
    }
    ...
  }
}
cpu.SetKeypad(cpu_keypad);

// cpu_chip8.cpp
void CpuChip8::SetKeypad(uint8_t* keys) {
  std::memcpy(keypad_state_, keys, 16);
}

Finally, our emulator is complete. We’ve built a complete CHIP-8 interpreter with support for writing a mono frame buffer, a system for converting that buffer to RGB images that get streamed to the GPU as textures, and have now corrected the timings and hooked up input.

I don’t know about you, but I learned quite a bit from building this project. It felt really good to build something from the ground up and finish it, and this project felt especially good because now I can actually play through these ROMs! With this completed, I’m one step closer to understanding the magic behind Super Mario World.

When last we left off we developed ourselves a chip-8 interpreter with working frame drawing. Now, we want to get those images out of our consoles and onto our screens.

We’re going to accomplish this using SDL, a library to blit graphics to the screen, capture user input, and play sounds. Setting up an SDL project can be a little tricky, so I recommended reading my post on it in order to get things building correctly.

There are many ways to get graphics onto the screen using SDL. For most games, complete images are not made on the CPU like we’re doing. However for emulation and (more commonly) video playback, a (potentially compressed) image sits on the CPU and needs to be uploaded to the GPU in order to be displayed. Once the image is on the GPU, we call it a “texture”, and we call this whole process “texture streaming”.

Now we have a particular image format that Image uses, but SDL provides a list of pixel formats that it understands. SDL_PIXELFORMAT_RGB24 seems like a fine enough format. Lets set up an SDLViewer class that will stream images of this format, and figure out later how to convert our frame buffer to RGB24.

// sdl_viewer.h

// RAII hardware-accelerated SDL Window.
// Optimized for RGB24 texture streaming.

class SDLViewer {
  public:
    // Width and height must be equal to the size of images uploaded
    // via SetFrameRGB24.
    SDLViewer(const std::string& title, int width, int height, int window_scale = 1);
    ~SDLViewer();

    // Renders the current frame, returns a list of all events.
    std::vector<SDL_Event> Update();

    // Assumes 8-bit RGB image with stride equal to width (no padding).
    void SetFrameRGB24(uint8_t* rgb24, int height);

  private:
    SDL_Window* window_ = nullptr;
    SDL_Renderer* renderer_ = nullptr;
    SDL_Texture* window_tex_ = nullptr;
};

We plan on using this class as an RAII SDL window that receives texture updates, and performs rendering. Our constructor will take a window scale because if we tried to draw a 64x32 image to the screen it would be very small.

SDLViewer::SDLViewer(const std::string& title, int width, int height, int window_scale) : 
      title_(title) {
  if(SDL_Init(SDL_INIT_VIDEO) < 0) {
    throw std::runtime_error(SDL_GetError());
  }
  // Create the SDL window with scaled dimensions.
  window_ = SDL_CreateWindow(title.c_str(), SDL_WINDOWPOS_UNDEFINED,
      SDL_WINDOWPOS_UNDEFINED, width * window_scale, height * window_scale, SDL_WINDOW_SHOWN);
  // Set up the hardware renderer and the texture we'll stream to.
  renderer_ = SDL_CreateRenderer(window_, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);
  SDL_SetRenderDrawColor(renderer_, 0xFF, 0xFF, 0xFF, 0xFF);

  window_tex_ = SDL_CreateTexture(renderer_, SDL_PIXELFORMAT_RGB24,
    SDL_TEXTUREACCESS_STREAMING, width, height);
}

SDLViewer::~SDLViewer() {
  SDL_DestroyTexture(window_tex_);
  SDL_DestroyRenderer(renderer_);
  SDL_DestroyWindow(window_);
  SDL_Quit();
}


std::vector<SDL_Event> SDLViewer::Update() {
  std::vector<SDL_Event> events;
  SDL_Event e;
  while (SDL_PollEvent(&e)) { events.push_back(e); }

  // Render our texture to the screen.
  SDL_RenderCopy(renderer_, window_tex_, NULL, NULL );
  SDL_RenderPresent(renderer_);

  return events;
}

void SDLViewer::SetFrameRGB24(uint8_t* rgb24, int height) {
  void* pixeldata;
  int pitch;
  // Lock the texture and upload the image to the GPU.
  SDL_LockTexture(window_tex_, nullptr, &pixeldata, &pitch);
  std::memcpy(pixeldata, rgb24, pitch * height);
  SDL_UnlockTexture(window_tex_);
}

We need to perform some boilerplate SDL initialization and destruction in the constructor and destructor. The Update method will present whatever the last image we sent to SDLViewer was. It also performs the important task of receiving input events.

SetFrameRGB24 is where we upload the texture to the GPU. It receives some image memory in the correct format along with a height. SDL_LockTexture returns some CPU memory that we need to copy to. It also returns the pitch, or row byte length, of the image. Once we copy the image to the alloted chunk we call SDL_UnlockTexture which will upload that image to the GPU as the new texture memory.

Now we ought to update our main loop to make use of this new window.

// main.cpp

void Run() {
  int width = 64;
  int height = 32;

  SDLViewer viewer("CHIP-8 Emulator",  width, height, /*window_scale=*/8);
   uint8_t* rgb24 = static_cast<uint8_t*>(std::calloc(
      width * height * 3, sizeof(uint8_t)));
  viewer.SetFrameRGB24(rgb24, height);

  CpuChip8 cpu;
  cpu.Initialize("/path/to/program/file");
  bool quit = false;
  while (!quit) {
    cpu.RunCycle();
    cpu.GetFrame()->CopyToRGB24(rgb24, /*r=*/255, /*g=*/0, /*b=*/0);
    viewer.SetFrameRGB24(rgb24, height);
    auto events = viewer.Update();

    for (const auto& e : events) {
      if (e.type == SDL_QUIT) {
        quit = true;
      }
    }
  }
}

We initialize our RGB24 image to a blank (zero/black) image. Note the size is not width * height, but width * height * 3. Since this is an RGB image, we have 3 channels. We upload the texture every cycle and present it the screen. With vsync this will cause the emulator to run very slowly, but this will be fixed when we go over timing. So now we just need to figure out what this RGB24 image format is and implement Image::CopyToRGB24.

When we make RGB images, we often interleave the red, green, and blue channels within memory. That is, adding 1 to a memory location no longer necessarily leads to the next pixel value.

0x000  :|RGBRGBRGB...----------------------------------------|
0x040*3:|RGBRGBRGB...                                        |
0x080*3:|RGBRGBRGB...                                        |
        ..
0x7C0*3:|RGBRGBRGB...----------------------------------------|

We now need some new terminology to discuss this. “Stride” (or “pitch”) often refers to the byte width of a row in an image. In this case, our RGB stride is 3 * width_px. We can also talk about stride as it relates to channels. To move from one red pixel to the next red pixel (the 0-dimension stride), we add 3. This is true of blue and green channels as well. Each individual value is still 8 bits (0 to 255) but a pixel as a whole now requires 3 values (the “24” in RGB24 is thus 3 channels * 8 bits). Okay, we should now have all we need to generate this image from our current mono image.

// image.cpp

 void Image::CopyToRGB24(uint8_t* dst, int red_scale, int green_scale, int blue_scale) {
  int cols = Cols();
  for (int row = 0; row < Rows(); row++) {
    for (int col = 0; col < cols; col++) {
      dst[(row * cols + col) * 3] = At(col, row) * red_scale;
      dst[(row * cols + col) * 3 + 1] = At(col, row) * green_scale;
      dst[(row * cols + col) * 3 + 2] = At(col, row) * blue_scale;
    }
  }
 }

We basically just iterate over the entire image and copy it over to dst, though for every pixel in the original image we need to copy three bytes over to the new image, one for each channel. Here we can take advantage of the fact that our original image values are either 0 or 1 to provide some color to the new image using some scaling parameters.

And just like that, we’ve got emulator output onto the screen! We’re even utilizing the GPU! Your program output should start looking at lot like the videos at the top of these articles, though likely a bit slower. In next article we’ll dive into timing in order to get everything running at the right speed.

In the last part of this series we built a CHIP-8 interpreter to execute all opcodes except for one, Dxyn - DRW Vx, Vy, nibble. To make this process easier, we’ll encapsulate our image memory and operations in an Image class. Our 64x32 frame will be represented as a single chunk of data in memory, where each pixel is a single byte:

0x000:|--------------------------------------------------------------|
0x040:|                                                              |
0x080:|                                                              |
0x0C0:|                                                              |
      ...
0x7C0:|--------------------------------------------------------------|

For our image above, we need to store three pieces of data: the number of rows, number of cols, and the starting address of the image memory (which malloc gives us). With this address pointing to the top-left corner of the memory, addressing individual pixels is fairly trivial. Some examples:

img[col=0, row=0] = img[0]
img[col=0, row=1] = img[width]
img[col=1, row=3] = img[3*width+1]

Okay, we should be ready to write the header.

// image.h

class Image {
  public:
    // Allocs and de-allocs in ctor and dtor.
    Image(int cols, int rows);
    ~Image();

    uint8_t* Row(int r);

    // Returns a pixel that can be changed.
    uint8_t& At(int c, int r);

    void SetAll(uint8_t value);

  private:
    int cols_;
    int rows_;

    uint8_t* data_;
};

What we need to pay attention to is that we’re dynamically allocating data that will be owned by this class. In a larger system, we may choose to use std::unique_ptr in conjuction with a specified allocation function, but here we’ll just malloc at construction time and free at destruction time.

// image.cpp

Image::Image(int cols, int rows) {
  data_ = static_cast<uint8_t*>(malloc(cols * rows * sizeof(uint8_t)));
  cols_ = cols;
  rows_ = rows;
}
Image::~Image() {
  free(data_);
}

uint8_t* Image::Row(int r) {
  return &data_[r * cols_];
}

uint8_t& Image::At(int c, int r) {
  return Row(r)[c];
}

void Image::SetAll(uint8_t value) {
  std::memset(data_, value, rows_ * cols_);
}

void Image::DrawToStdout() {
  for (int r = 0; r < rows_; r++) {
    for (int c = 0; c < cols_; c++) {
      if (At(c,r) > 0) {
        std::cout << "X";
      } else {
        std::cout << " ";
      }
    }
    std::cout << std::endl;
  }
  std::cout << std::endl;
}

I like to use rows and columns here rather than x and y because I rarely forgot what a row is, but I often forget what an “x” is. Our At function returns a uint8_t& so that we can use it for both getting and setting individual pixels. This is bad for encapsulation, but is common among image APIs. We also added a handy DrawToStdout so that we can view the image in our console until we set up graphical output later. Now we can add our frame to the CPU and work out an implementation.

// cpu_chip8.h

class CpuChip8 {
 public:
  constexpr innt kFrameWidth = 64;
  constexpr innt kFrameHeight = 32;

  CpuChip8() : frame_(kFrameWidth, kFrameHeight) {}
  ...
 private:
  ...
  Image frame_;
};

Finally we’ll talk about how CHIP-8 performs drawing. All drawing is done in terms of an XOR operation of a sprite onto the current and only frame buffer. All sprites are defined as images with a depth of 1-bit (each pixel on or off), a width of 8, and a variable height. The width limitation is due to the fact that each pixel in the sprite is only a single bit. Let’s take a look at the fontset earlier and try to decode a digit.

0xF0, 0x90, 0x90, 0x90, 0xF0, // 0

0xF0 is 1111 0000 -> XXXX
0x90 is 1001 0000 -> X  X
0x90 is 1001 0000 -> X  X
0x90 is 1001 0000 -> X  X
0xF0 is 1111 0000 -> XXXX

Pretty cool! So when I said that drawing is done as an XOR operation, the only way to remove a sprite from the screen is to draw over it (1 ⊕ 1 is 0). This is why you often see flickering in CHIP-8 programs, the sprite is continuously drawing and un-drawing to depict movement.

Okay we’re just about ready to write a sprite drawing function. We’re going to need a starting point and a sprite (a chunk of memory). Since sprite height is variable, we’ll receive that parameter too. One thing that took me quite a long time to debug is that the CHIP-8 interperter supports drawing out of bounds. When that happens, the draw continues as it wraps around. This also works for the starting coordinates of the (so drawing at 255,255 for 15 rows is perfectly valid). Also, the interpreter needs to receive back whether or not a pixel was turned off (this is often used for collision detection).

// image.cpp

// Returns true if the new value unsets the pixel.
bool Image::XOR(int c, int r, uint8_t val) {
  uint8_t& current_val = At(c, r);
  uint8_t prev_val = current_val;
  current_val ^= val;
  return current_val == 0 && prev_val > 0;
}

bool Image::XORSprite(int c, int r, int height, uint8_t* sprite) {
  // Wrap around the screen as we draw.
  bool pixel_was_disabled = false;
  for (int y = 0; y < height; y++) {
    int current_r = r + y;
    while (current_r >= rows_) { current_r -= rows_; }
    uint8_t sprite_byte = sprite[y];
    for (int x = 0; x < 8; x++) {
      int current_c = c + x;
      while (current_c >= cols_) { current_c -= cols_; }
      // Note: We scan from MSbit to LSbit
      uint8_t sprite_val = (sprite_byte & (0x80 >> x)) >> (7-x);
      pixel_was_disabled |= XOR(current_c, current_r, sprite_val);
    }
  }
  return pixel_was_disabled;
}

We need to take care to extract the bits as either 1 or 0. Since our Image class supports [0..255] our XORs could get messy without this restriction. With this in place our CPU instruction is pretty easy, we just need to extract the parameters needed to call XORSprite.

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenDRAW(uint8_t reg_x, uint8_t reg_y, uint8_t n_rows) {
  return [this, reg_x, reg_y, n_rows]() {
    uint8_t x_coord = v_registers_[reg_x];
    uint8_t y_coord = v_registers_[reg_y];
    bool pixels_unset = frame_.XORSprite(x_coord, y_coord, n_rows,
      memory_ + index_register_);
    v_registers_[0xF] = pixels_unset;
    NEXT;
  };
}

At this stage you should be able to execute some ROMs! Set up a call to DrawToStdout after a cycle execution and watch the output in your console. You’ll have to run a program that doesn’t expect any user input, however.

In the next part of this series, we’ll hook up SDL to get some actual graphics onto the screen. We’ll also finally wire up the input!

For several reasons, emulation has always fascinated me. A program that executes other programs sounds like such a cool concept. It really feels like you’re getting your money’s worth out of writing it! Beyond that, it definitely feels like you’re building a computer within software. I really enjoyed learning about computer architecture and writing some basic HDL code, but emulation is a much more straightforward way of achieving a similar feeling of generating a machine. I’ve also always had this goal of knowing exactly how Super Mario World worked, ever since I first saw it as a kid. Because of this, writing a SNES/SFC emulator has been on my mind for a while. I decided recently that it was time to take a step forward towards making this happen.

So let’s take a look at writing an emulator. A simple, but complete example would involve CHIP-8.

CHIP-8 is actually a programming language. It’s really simple too, there’s only 35 opcodes. To write an interpreter for it, we pretty much just need to write a program that can execute all 35 different instructions. The emulation aspect of this comes from the bits you wouldn’t normally find in a programming language interpreter. We need a way to display graphics, process user input, play audio, and we need to simulate the hardware mechanisms of a CHIP-8 machine. Things like registers and memory need to be taken into account during execution, and we also need to be careful about timing.

Let’s start! For this project, we’ll be using C++. This should be fairly trivial to translate into other languages. If you want to take a look at the complete source, see the project repository.

First, a basic main loop. We’ll ignore emulating timing for now.

// main.cpp

void Run() {
  CpuChip8 cpu;
  cpu.Initialize("/path/to/program/file");
  bool quit = false;
  while (!quit) {
    cpu.RunCycle();
  }
}

int main(int argc, char** argv) {
  try {
    Run();
  } catch (const std::exception& e) {
    std::cerr << "ERROR: " << e.what();
    return 1;
  }
}

Our CpuChip8 class will encapsulate the state of our virtual machine and interpreter. Now if we implement RunCycle and Initialize we’ll have ourselves a basic emulator skeleton. We now need to discuss the phsyical system we’re emulating.

Our CHIP-8 system will be the Telmac 1800. We’ve got ourselves a pool of 4K of memory, a 64x32 1-bit display, and the ability to beep. Nice. The CHIP-8 interpreter itself is implemented via a virtual machine. We need to keep track of a stack, sixteen 8-bit registers (named V0 through VF), a 12-bit index register (named I), a program counter, two 8-bit timers, and a 16-frame stack.

The canonical memory map looks like this:

0x000 |--------------------|
      | Interpreter memory |
      |                    |
0x050 | Built-in fontset   |
0x200 |--------------------|
      |                    |
      |                    |
      | Program memory     |
      | and dynamic allocs |
      |                    |
      |                    |
0xFFF |--------------------|

You’ll notice there’s no explicit stack here. The program actually doesn’t have a stack to address, that’s only used by the interpreter to implement jumping to functions and back. With this in mind we can draw up a header.

// cpu_chip8.h

class CpuChip8 {
 public:
  public Initialize(const std::string& rom);
  void RunCycle();

 private:
  // Fills out instructions_.
  void BuildInstructionSet();

  using Instruction = std::function<void(void)>;
  std::unordered_map<uint16_t, Instruction>> instructions_;

  uint16_t current_opcode_; 

  uint8_t memory_[4096];  // 4K
  uint8_t v_register_[16];

  uint16_t index_register_;
  // Points to the next instruction in memory_ to execute.
  uint16_t program_counter_;

  // 60Hz timers.
  uint8_t delay_timer_;
  uint8_t sound_timer_;

  uint16_t stack_[16];
  // Points to the next empty spot in stack_.
  uint16_t stack_pointer_;

  // 0 when not pressed.
  uint8_t keypad_state_[16];
};

We use excplicit integer types to ensure values are over/underflowed correctly. We need to use 16-bit types for 12-bit values. We also have 16 digital input keys, which we store as either on or off within this class. When we hook up input, we’ll find a way to feed that into the class between cycles. The opcode is made easy by the fact that all CHIP-8 instructions are 2 bytes long.

So that gives us 0xFFFF=64k possible instructions (though many are unused). We can actually store every possible instruction in a map so that when we fetch an opcode we are able to immediately execute it by calling the associated Instruction in instructions_. Since we don’t bind much data to the functions, which should be able to fit the entire instruction map in cache!

Our Initialize function is where to set up the memory map described above:

// cpu_chip8.cpp

CpuChip8::Initialize(const std::string& rom) {
  current_opcode_ = 0;
  std::memset(memory_, 0, 4096);
  std::memset(v_registers_, 0, 16);
  index_register_ = 0;
  // Program memory begins at 0x200.
  program_counter_ = 0x200; 
  delay_timer_ = 0;
  sound_timer_ = 0;
  std::memset(stack_, 0, 16);
  stack_pointer_ = 0;
  std::memset(keypad_state_, 0, 16);
  
  uint8_t chip8_fontset[80] =
  { 
    0xF0, 0x90, 0x90, 0x90, 0xF0, // 0
    0x20, 0x60, 0x20, 0x20, 0x70, // 1
    0xF0, 0x10, 0xF0, 0x80, 0xF0, // 2
    0xF0, 0x10, 0xF0, 0x10, 0xF0, // 3
    0x90, 0x90, 0xF0, 0x10, 0x10, // 4
    0xF0, 0x80, 0xF0, 0x10, 0xF0, // 5
    0xF0, 0x80, 0xF0, 0x90, 0xF0, // 6
    0xF0, 0x10, 0x20, 0x40, 0x40, // 7
    0xF0, 0x90, 0xF0, 0x90, 0xF0, // 8
    0xF0, 0x90, 0xF0, 0x10, 0xF0, // 9
    0xF0, 0x90, 0xF0, 0x90, 0x90, // A
    0xE0, 0x90, 0xE0, 0x90, 0xE0, // B
    0xF0, 0x80, 0x80, 0x80, 0xF0, // C
    0xE0, 0x90, 0x90, 0x90, 0xE0, // D
    0xF0, 0x80, 0xF0, 0x80, 0xF0, // E
    0xF0, 0x80, 0xF0, 0x80, 0x80  // F
  };
  // Load the built-in fontset into 0x050-0x0A0
  std::memcpy(memory_ + 0x50, chip8_fontset, 80);

  // Load the ROM into program memory.
  std::ifstream input(filename, std::ios::in | std::ios::binary);
  std::vector<uint8_t> bytes(
         (std::istreambuf_iterator<char>(input)),
         (std::istreambuf_iterator<char>()));
  if (bytes.size() > kMaxROMSize) {
    throw std::runtime_error("File size is bigger than max rom size.");
  } else if (bytes.size() <= 0) {
    throw std::runtime_error("No file or empty file.");
  }
  std::memcpy(memory_ + 0x200, bytes.data(), bytes.size());

  BuildInstructionSet();
}

Don’t worry about trying to read that file loading code, the C++ iostream library is kind of ridiculous. The gist of it here is that we set everything to 0 and load things into memory that need to be loaded. The fontset here is a series of 16 built-in sprites that programs can reference as they want. We’ll go over how that memory forms sprites later on when we worry about graphics. Our goal is that once Initialize complete we’re set up to execute a user program.

Let’s build out a basic RunCycle so that we have a better idea of out to write BuildInstructionSet. If you remember any basic computer architecture, a cycle has a few phases. First you fetch the instruction, then you decode it, then you execute it.

// cpu_chip8.cpp

void CpuChip8::RunCycle() {
  // Read in the big-endian opcode word.
  current_opcode_ = memory_[program_counter_] << 8 |
    memory_[program_counter_ + 1];

  auto instr = instructions_.find(current_opcode_);
  if (instr != instructions_.end()) {
    instr->second();
  } else {
    throw std::runtime_error("Couldn't find instruction for opcode " +
      std::to_string(current_opcode_));
  }

  // TODO: Update sound and delay timers. 
}

This is pretty much just a map lookup to find the function to execute. The one weird bit here is how we read in the next opcode. CHIP-8 uses a big-endian archicture, which means the most-significant part of the word comes first, followed by the least significant part of the word. This is reversed in modern x86-based systems.

Memory location 0x000: 0xFF 
Memory location 0x001: 0xAB

Big endian interpretation:    0xFFAB
Little endian interpretation: 0xABFF

Note that we don’t alter the program counter within RunCycle. This is done on a function-by-function base, so we leave that to the implementation of the particular Instruction. Also, since we chose to define Instruction as a function pointer without any arguments, we’re going to have to bind those to the function itself. This requires more work in the initial set-up, but means we completely remove the instruction-decode phase on RunCycle.

Let’s dig into the meat of the interpreter, BuildInstructionSet. I wont list the implementations for every function here, but you can find that in the repository for this project. I highly recommend coding this alongside something like Cowgod’s technical reference.

// cpu_chip8.cpp

#define NEXT program_counter_ += 2
#define SKIP program_counter_ += 4

void CpuChip8::BuildInstructionSet() {
  instructions_.clear();
  instructions_.reserve(0xFFFF);

  instructions_[0x00E0] = [this]() { frame_.SetAll(0); NEXT; }; // CLS
  instructions_[0x00EE] = [this]() {
    program_counter_ = stack_[--stack_pointer_] + 2;  // RET
  };

  for (int opcode = 0x1000; opcode < 0xFFFF; opcode++) {
    uint16_t nnn =  opcode & 0x0FFF;
    uint8_t kk =    opcode & 0x00FF;
    uint8_t x =     (opcode & 0x0F00) >> 8;
    uint8_t y =     (opcode & 0x00F0) >> 4;
    uint8_t n =     opcode & 0x000F;
    if ((opcode & 0xF000) == 0x1000) {
      instructions_[opcode] = GenJP(nnn);
    } else if ((opcode & 0xF000) == 0x2000)) {
      instructions_[opcode] = GenCALL(nnn);
    }
    // ...
}

Each instruction may encode some parameters, which we decode and use when needed. We could use std::bind here to generate the std::functions, in this case I chose to define Gen[INSTRUCTION_NAME] functions which will return the functions as lambdas with all of the data bound. Lets look at some of the more interesting functions:

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenJP(uint16_t addr) {
  return [this, addr]() {  program_counter_ = addr; };
}

When we JP to an address, we just set the program counter to the address. That’ll cause the next cycle to execute the instruction at that point.

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenCALL(uint16_t addr) {
  return [this, addr]() {
    stack_[stack_pointer_++] = program_counter_;
    program_counter_ = addr;
  };
}

We do the same thing when we CALL a function at an address. Here, however, we need to provide a way to later return from the callsite. To do this, we store the current program counter onto the stack.

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenSE(uint8_t reg, uint8_t val) {
  return [this, reg, val]() {
    v_registers_[reg] == val ? SKIP : NEXT;
  };
}

SE mean “skip if the immediate value is equal to the value in the provided register”. The instruction receives the V register to dereference, and we set the program counter accordingly.

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenADD(uint8_t reg_x, uint8_t reg_y) {
  return [this, reg_x, reg_y]() {
    uint16_t res = v_registers_[reg_x] += v_registers_[reg_y];
    v_registers_[0xF] = res > 0xFF; // set carry
    v_registers_[reg_x] = res;
    NEXT;
  };
}
CpuChip8::Instruction CpuChip8::GenSUB(uint8_t reg_x, uint8_t reg_y) {
  return [this, reg_x, reg_y]() {
    v_registers_[0xF] = v_registers_[reg_x] > v_registers_[reg_y]; // set not borrow
    v_registers_[reg_x] -= v_registers_[reg_y];
    NEXT;
  };
}

When adding or subtracting registers, we need to keep track of overflow. If we detect it, we set VF.

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenLDSPRITE(uint8_t reg) {
  return [this, reg]() {
    uint8_t digit = v_registers_[reg];
    index_register_ = 0x50 + (5 * digit);
    NEXT;
  };
}

Our sprite loading function is fairly trivial, it is used by the program to figure out where a certain digit is within the built-in fontset. Remember we stored our fontset at 0x50 and each character is 5-bytes wide. So we set I to 0x50 + 5 * digit.

// cpu_chip8.cpp

CpuChip8::Instruction CpuChip8::GenSTREG(uint8_t reg) {
  return [this, reg]() {
    for (uint8_t v = 0; v <= reg; v++) {
      memory_[index_register_ + v] = v_registers_[v];
    }
    NEXT;
  };
}
CpuChip8::Instruction CpuChip8::GenLDREG(uint8_t reg) {
  return [this, reg]() {
    for (uint8_t v = 0; v <= reg; v++) {
      v_registers_[v] = memory_[index_register_ + v];
    }
    NEXT;
  };
}

When we interface directly with memory, the user provides the maximum register they’d like to use. For instance if they want to load registers V0, V1, V2 with the values stored sequentially in MEM[I] they’d pass in V2 after setting up I.

With that, we’ve got ourselves a CHIP-8 interpreter! Sure, there’s no sound or graphics hooked up, but as long as you don’t use those functions you should be able to execute some basic test ROMs. In the next part of this series, we’ll look at drawing, the most complex operation that the interpreter performs.

macOS and Linux

Let’s start with a POSIX environment like macOS or a Linux distribution. If you’re running on macOS you wont have a package manager by default, but Homebrew is a common choice and easy to use. Once installed. Once your environment has a package manager you can go ahead and install all of the SDL development libraries.

$ brew install sdl2 sdl2_image sdl2_mixer sdl2_net sdl2_ttf

Great, now we can begin. We’ll use a simple Makefile for our project:

CXX=g++
SDL2CFLAGS=-I/usr/local/include/SDL2 -D_THREAD_SAFE

CXXFLAGS=-O2 -c --std=c++14 -Wall $(SDL2CFLAGS)
LDFLAGS=-L/usr/local/lib -lSDL2

exec: main.o
	$(CXX) $(LDFLAGS) -o exec main.o

main.o: main.cpp
	$(CXX) $(CXXFLAGS) main.cpp

The process here is simple, we build our main.cpp file into an object using our SDL2 headers, then link it against the SDL2 dynamic libraries. In main.cpp we can access SDL_* functions with a simple #include <SDL2/SDL.h> directive. The header files will tell the compiler how the API works, and the dynamic libraries will tell the linker how they’re implemented.

But where did we get the SDL2CFLAGS and LDFLAGS values? Well we installed the SDL libraries we also got access to sdl2-config. You can use it like this:

$ sdl2-config --cflags
$ sdl2-config --libs

It’s a handy way to find out where your package manager installed the files. Now, assuming you’ve got a C++ compiler, you can go ahead and run make exec to build your program.

Windows

So we’ve got an SDL program that we wrote on macOS or Linux, and we want to share it with our friends running Windows. How do we make a build for them? Well we could install MinGW which provides us with g++ and make, but for my project I found that the version of g++ included here was lacking. It didn’t support std::thread after installation. So much for standardization.

That leaves us with Visual Studio, specifically Visual C++, or MSVC. Microsoft isn’t good with naming these things. It’s an IDE and a C++ compiler, tightly coupled. The process here is a little finnicky. You need to open your SDL project as a project without a “Solution” (which is going to replace our Makefile on Windows) so that VS can generate one for us. Skip past all of the dialogue (when prompted, indicate you’re building a command line app) so that we can start configuring things.

Without a package manager on Windows you need to manually make your way here and download the development libraries for Visual C++. Unzip those to a folder you’ll remember later. As a tip, alter the SDL2 include folder structure to place all headers in a dir called SDL2. For some reason the Windows distribution of this package doesn’t do that, which would mean your includes would look like #include <SDL.h> instead of #include <SDL2/SDL.h>.

Okay, last step. We need to tell VS how to use the libraries we just downloaded. Go to Project -> Properties -> Configuration Properties -> VC++ Directories -> Include Directories -> Edit and add the path to the include subdir from the SDL2 package you unzipped earlier. These are the header files. Then go to Linker -> Additional Dependencies -> Edit and add SDL2.lib; SDL2main.lib;. These are the dynamic libraries, but we still need to tell VS where those blobs are, so go to VC++ Directories -> Include Directories -> Edit and add the path to the lib/x64 (or x86 for 32-bit builds) subdir from the SDL2 package you unzipped earlier. Finally, save everything and build your project.

With that, you can sleep safely knowing your project will build just about anywhere. When you want to release on macOS or Linux, use the Makefile, and when you want to release on Windows, use the Visual Studio solution.

For me I was always perplexed by malloc. It’s a simple function, you ask for memory and it gives it to you. But how could a C function do that? What on Earth does it mean to allocate memory? Isn’t it all there, sitting on the bus, just waiting for us to issue some good ‘ol MOV instructions? Even worse, it is used everywhere. Even if you aren’t using C there’s a good chance you’re using malloc, every time you create a new object in a language implemented in C, like cpython for instance. It isn’t the only way to acquire memory, but it sure is a popular one.

So let’s take a look at malloc, it can’t be that complicated right? Here’s a simple malloc:

void* malloc(size_t size) {
  return sbrk(size);
}

From this you may be able to figure out what sbrk does. It gives us a chunk of memory. Specifically it allocates program heap memory of a given size and returns a pointer to it. What exactly is “it” that is being pointed to? Let’s start printing stuff and find out.

void* a = sbrk(0);
void* b = sbrk(100);
void* c = sbrk(0);
printf("[a: %p]\n[b: %p]\n[c: %p]\n", a, b, c);

> [a: 0x1065b4064]
  [b: 0x1065b4064]
  [c: 0x1065b40c8]

Looks like b is the same as a and c is b + 100 bytes. So sbrk returns a pointer to whatever the last ‘tip’ of the program heap is, and if we give it a size in bytes it’ll move that ‘tip’ up. Meaning if we sbrk ourselves that 100 byte chunk we can do whatever we want with it knowing that the next time we sbrk ourselves some more memory, it’ll be 100 bytes farther along.

This may not be too satisfying, we’ve replaced one magical function with another. However, in this case sbrk is a system call. It’s going to jump the CPU over to some assembly to execute (the instruction set implemented by your CPU is very likely to have a set of functions for interfacing with memory), at least now we’re talking to the kernel instead of wondering what’s going in the the C standard library.

So that’s malloc, simple right? Well judging from the length of this article you can probably deduce otherwise. There’s more here, and for two reasons:

1. sbrk is absolutely ancient and super-deprecated. In fact if you run these snippets on macOS you’re going to get tons of warnings (but hey, it still works!). It doesn’t work with virtual memory and it isn’t thread-safe. However, its API is very simple to use and malloc at one point in time very likely was implemented using sbrk.
2. This implementation of malloc is incorrect. The first reason why, which you may be able to guess, is that sbrk can fail. Memory is a finite resource.

According to man sbrk, the call can return -1 if it fails, but malloc is supposed to return NULL. This is fixed easily enough.

void* malloc(size_t size) {
  void* chunk_start = sbrk(size);
  return chunk_start == (void*)-1 ? NULL : chunk_start;
}

One more thing. malloc(0) has special behavior, it needs to return NULL as well, otherwise you’d be able to get a pointer back from malloc that you didn’t actually allocate, and that would be weird.

void* malloc(size_t size) {
  if (size == 0) return NULL;
  void* chunk_start = sbrk(size);
  return chunk_start == (void*)-1 ? NULL : chunk_start;
}

Great, now the user can actually know whether their memory request was fulfilled. We’re still missing something though, the result of our malloc doesn’t work with free. What is free exactly?

According to man free, free will remove the allocation (“free”ing the space) from an input pointer that was previously returned from malloc. Now sbrk has a feature where when a negative input is passed to it, it will move the tip of the heap down instead of up. So effectively it allows us to push and pop from the program heap, because the heap is a stack and computer terminology is silly.

Unfortunately this isn’t enough for us. Imagine a user does the following:

void* a = malloc(500);
// We can now call sbrk(-500) to free a.
void* b = malloc(1000);
// But how do we free a from here?

This is the age-old problem of trying to delete something from the middle of the stack. We could pop everything off of the stack until we reach the memory we’re trying to delete (storing it somewhere else, we could even use the disk), then pop the item to delete, then push everything else back onto the stack. This would be miserably slow. We could also abandon the stack mentality and just use memcpy to copy over the old bytes. This would also be very slow, usually we expect free to take an insignificant amount of time to complete. In either of these cases, we’ve created a new problem: when shifting all of the old memory to utilize the newly free’d space, all of the pointers in the program refering to that old memory would be invalidated.

It looks like we’re going to have take matters into our own hands. Maybe in the future we’ll have more memory than we know what to do with and never free anything. Until then we’ll need to do something clever. We have one thing going for us though, malloc always returns a pointer to contiguous memory. If we have a single “hole” of freed memory in the heap large enough to use somewhere, we can use it. It’s simply a matter, then, of us keeping track of the allocated chunks (and the “holes” created by freeing those chunks) ourselves.

So maybe it was wrong to say earlier that sbrk “allocates” memory for us, because now it seems like it just moves some pointer and tells us where it used to be. The actual “allocation” part of malloc is something we’ll have to implement. Let’s set up a general outline of what we want to accomplish.

typedef struct chunk {
  size_t size;  // size of user-accessible memory.
  struct chunk* next;
  bool allocated;
} Chunk;

Chunk* heap;

void* chunk_data(Chunk* chunk) {
  // Adding 1 to a Chunk* will get us to the part of memory
  // directly after the fields.
  return chunk ? chunk+1 : NULL;
}

// Returns the Chunk corresponding to the chunk's data.
Chunk* chunk_metadata(void* ptr) {
  return ptr ? (Chunk*)ptr-1 : NULL;
}

 // TODO: Do the hard part.
Chunk* find_or_reserve_chunk(size_t size);

void* malloc(size_t size) {
  if (size == 0) return NULL;
  return chunk_data(find_or_reserve_chunk(size));
}

void free(void* ptr) {
  Chunk* chunk = chunk_metadata(ptr);
  if (!chunk) return;
  chunk->allocated = false;
}

We want some way to model the heap, so a linked list sounds simple enough. We could extend it to create an actual stack, but it isn’t really needed here. Normally we’d implement a linked list using malloc, but we’re implementing malloc so we can’t really use that, can we? So let’s just sneak our data structure in with the user data. The general idea is that for every chunk of memory allocated by malloc, the we store a few bytes (specifically sizeof(Chunk)) of metadata about that chunk right beforehand. We could store a pointer in Chunk to the actual user memory, but since the memory is contiguous we can easily compute where the metadata ends and the user data begins. freeing then becomes super easy, we can just get the metadata and mark that it’s no longer allocated. The hard part is using that information.

// Allocates a new chunk right after 'prev'.
Chunk* allocate_chunk(Chunk* prev, size_t size);

Chunk* find_or_reserve_chunk(size_t size) {
  Chunk* prev_chunk = NULL;

  // Initialize the heap if necessary.
  if (!heap) {
    heap = allocate_chunk(prev_chunk, size);
    return chunk_data(heap);
  }

  // Scan the heap for holes large enough for the chunk we want.
  Chunk* chunk = heap;
  while (chunk) {
    if (!chunk->allocated && chunk->size >= size) break;
    prev_chunk = chunk;
    chunk = chunk->next;
  }

  if (chunk) {
    chunk->allocated = true;
  } else {
    chunk = allocate_chunk(prev_chunk, size);
  }

  return chunk;
}

From here you can devise faster ways of doing this. The memory-speed tradeoff here is real, you can avoid scanning the heap every time by reserving a portion of the start of the heap for a hash map storing holes by size requirement, but then of course you’ve got less of the heap for the user. You also need to consider the size of this portion. You won’t be able to grow it, since then you’d be invalidating user pointers.

If speed is less of a concern, you’d want to scan the entire heap rather than stopping at the first available hole. With the above implementation the user would often get back chunks of memory where the allocation is actually greater than they requested. If you scan the entire heap, you can look for the smallest hole that fits the requirements. Another way of doing this is to terminate the chunk to always fit the requested size, creating a new chunk for the leftover data. You’d have to make sure whatever leftover has enough room for the metadata fields.

This is why the data stored in the pointer returned from malloc is uninitiliazed, it may have been a chunk from some previously requested memory.

The last bit is where we actually build up the heap model:

Chunk* allocate_chunk(Chunk* prev, size_t size) {
  Chunk* chunk = (Chunk*)sbrk(size + sizeof(Chunk));
  if (chunk == (Chunk*)-1) {
    return NULL;
  }

  if (prev) {
    prev->next = chunk;
  }

  chunk->size = size;
  chunk->allocated = true;
  chunk->next = NULL;
  return chunk;
}

We take in the last chunk in order to avoid needing to scan the heap again. We need to be careful about asking sbrk for enough memory, since we’re also storing the Chunk metadata. Because sbrk returns -1 as a pointer on failure, we have to do some weird casting in order to generate the literal for comparison.

There are a few other memory allocation functions like calloc and realloc, but if you’ve gotten this far you can probably determine how to implement those in terms of malloc and free. A fun challenge would be figuring out how to implement aligned_alloc. There’s also the issue of thread-safety, you can imagine things will go very poorly if two separate threads are operating on heap, a simple way to reconcile this is to protect it with a mutex.

Having written this I feel at ease once again, malloc makes sense and all is right in the world. The trick was just learning how to model the heap within the heap itself. This is a very basic and slow implementation, but it gets us up and running and can be used for accessing dynamic memory in programs, which is all the header files really ever promised us.

My pixels have gone, my pixels have vanished! Perhaps they weren’t even there to begin with. Won’t someone teach me the orthopraxy?

— Recorded 16 seconds pre-coffee break. Subject: a frustrated coder.

So you’ve decided to build a game or animation engine, and you’re interested in rendering the whole thing in React, maybe even with TypeScript. After a few attempts, you’ve realized that building the rendering loop is trickier than you thought. Or at least, that was what happened to me as I was working on my Game Boy emulator. After a little research, I’ve come to a solution that I’m happy with. Let’s see how to get there.

The first thing we need is a way to get graphics onto the screen. For the purposes of this demonstration, we’ll be generating our own ImageData and putting the pixels onto a canvas using putImageData(), though a similar structure could be used if you were to use WebGL. If you take a look at Phil Nash’s post on the subject, you’ll learn how to go about rendering a canvas element in React the correct way. The first step is to create a component whose sole purpose to render a canvas to the DOM.

interface PureCanvasProps {
  contextRef: Function;
}

class PureCanvas extends Component<PureCanvasProps, any> {
  shouldComponentUpdate() {
    return false;
  }

  render() {
    return (
      <canvas
        width="160"
        height="144"
        ref={node =>
          node ? this.props.contextRef(node.getContext('2d')) : null
        }
      />
    );
  }
}

There are a few interesting things to note here. The first is that we’ve turned off all updating for this component by using shouldComponentUpdate(). We want that because it doesn’t make sense for us to ever re-render the <canvas> element. Unlike most React or HTML tags, its state is not determined by its properties or children, but by a context exposed by the element itself. This means we can’t update it just by passing the new ImageData in as a property. This presents an interesting problem, because we are unable to actually render to the canvas in any React component rendering method. We do accept one property for PureCanvas, a contextRef function. This is how we’re going to manipulate the canvas data, by passing in a function that will wrap the canvas context into the state of a parent component. Note: getContext() is a method exposed by the canvas element to gain access to the canvas from a reference. You can read more about it here.

Fantastic, now we’ve got a target for our rendering, let’s see how to use it in a loop!

interface GraphicsRendererState {
  frameId: number;
  engine: Engine | null;
}

class GraphicsRenderer extends Component<any, GraphicsRendererState> {
  state: GraphicsRendererState = {
    frameId: 0,
    engine: null,
  };

  componentDidMount() {
    this.setState({
      frameId: requestAnimationFrame(this.updateAnimationState),
    });
  }

  componentDidUpdate() {
    if (this.state.engine) {
      this.state.engine.drawFrame();
    }
  }

  componentWillUnmount() {
    cancelAnimationFrame(this.state.frameId);
  }

  saveContext = (ctx: CanvasRenderingContext2D) => {
    this.setState({
      engine: new Engine(ctx),
    });
  };

  updateAnimationState = () => {
    const { engine } = this.state;
    if (engine) {
      // After tick() is finished, we have a frame to draw
      engine.tick();
    }
    // This will trigger componentDidUpdate(), giving us a chance to draw
    this.setState({frameId: requestAnimationFrame(this.updateAnimationState)});
  };

  render() {
    return <PureCanvas contextRef={this.saveContext} />;
  }
}

Let’s walk through this! First, the context. GraphicsRenderer::render() returns only the PureCanvas described earlier, and for the contextRef function, we pass in a function to wrap the CanvasRenderingContext2D into the state of GraphicsRenderer. We’re doing this through the use of an Engine class, which acts as your game or animation engine. When we initialize GraphicsRenderer, we don’t have any canvas context and so engine is null. To allow for this this, in GraphicsRendererState we use union types to tell TypeScript that we want engine to be nullable.

So that takes care of wrapping the canvas context, how do we actually set up our frame loop? In GraphicsRenderer, we do that through the use of requestAnimationFrame(). If you’ve done graphics programming before, this function is sort of like a wait frame function or an acquire frame function, depending on the API. What that means is that requestAnimationFrame() is an incredibly useful function that handles the work of synchronizing code to the refresh rate of the browser (60Hz). It’s pretty simple too, you pass it a callback to execute and it returns a unique ID to let you cancel any requests later. It’s important to note that this isn’t called on an interval, you have to call requestAnimationFrame() again on every callback. This allows it work just fine even if your game can’t hit 60fps.

In our updateAnimationState() callback, we tick the engine (computing an image to draw) and request another frame, setting the component state to that frame ID. This setState() call is useful, as it will trigger a component update, giving us a chance to actually draw the image. Since we’ve been keeping track of the frame request ID, let’s make sure to cancel any outbound request whenever we unmount the component. We do this with cancelAnimationFrame().

Now that we’ve got our loop built, let’s make a simple engine and test it.

let z = 0;

class Engine {
  ctx: CanvasRenderingContext2D;
  frameBuffer: ImageData;
  constructor(ctx: CanvasRenderingContext2D) {
    this.ctx = ctx;
    this.frameBuffer = ctx.createImageData(ctx.canvas.width, ctx.canvas.height);
  }
  tick = () => {
    // Iterate through every pixel
    for (let i = 0; i < this.frameBuffer.data.length; i += 4) {
      // Modify pixel data
      this.frameBuffer.data[i + 0] = z; // R value
      this.frameBuffer.data[i + 1] = z; // G value
      this.frameBuffer.data[i + 2] = z; // B value
      this.frameBuffer.data[i + 3] = 255; // A value
    }
    z++;
    if (z > 255) {
      z = 0;
    }
  };
  drawFrame = () => {
    this.ctx.putImageData(this.frameBuffer, 0, 0);
  };
}

This will render a square that slowly becomes lighter every tick. Now for the moment of truth, the performance. We can test for this by running a system trace right within the Chrome DevTools.

The target frame time for 60fps is 16.7ms, and it looks like we’re hitting that! But something stands out in this trace, the setState() calls. Every frame, we’re spending a little over 2ms just on setState()! That time will become extremely valuable as you flesh out your engine. We need to fix this.

So let’s take a look at how we’re actually using setState(). Since PureCanvas never updates, we’re only using it in GraphicsRenderer. Even then, we’re really only changing the state to account for the animation frame ID updates, and we’re only using those so that we can cancel requests once the component unmounts! Let’s see if we can get by without it.

interface GraphicsRendererState {
  engine: Engine | null;
}

class GraphicsRenderer extends Component<any, GraphicsRendererState> {
  state: AnimationState = {
    engine: null,
  };

  componentDidMount() {
    requestAnimationFrame(this.updateAnimationState)
  }

  saveContext = (ctx: CanvasRenderingContext2D) => {
    this.setState({
      engine: new Engine(ctx),
    });
  };

  updateAnimationState = () => {
    const { engine } = this.state;
    if (engine) {
      // After tick() is finished, we have a frame to draw.
      engine.tick();
      // Drawing it right now is fine, let's do it!
      engine.drawFrame();
    }
    requestAnimationFrame(this.updateAnimationState);
  };

  render() {
    return <PureCanvas contextRef={this.saveContext} />;
  }
}

We’ve nixed all state modifications except for the initial engine creation, which’ll only happen once. We’re also drawing the frame directly after the engine tick. Since the callback is synchronized, drawing the frame then will be as well! Since we don’t keep track of the frame ID, we’re no longer cancelling the animation frame. How does this affect us? Well, in our setup we will only ever have one outbound request at a time, meaning there will be a chance upon unmount that updateAnimationState() is called after unmount. Depending on how you structure your engine, you may need to add a little bit of additional error checking, as the canvas context will no longer be valid. If the only time your component unmounts is when the app closes, then you don’t even need to worry about this. So how is the performance now?

We’ve saved so much time! Since our component isn’t really updating, we have ample time for our engine to do computations.

All good now, right? Nope! Think about what is happening here and what we’ve learned from the performance traces. requestAnimationFrame() gives you a chance to draw a frame on time. What are we doing though? We’re doing some heavy work to create a frame, then we draw it, then we request another time to draw. This means that, while the start of the animation callback is synchronized, the end is not! Our engine ticks could take a variable amount of time, and we only draw once they are over. This means that the image displayed within the canvas isn’t necessarily displayed for the same amount of time every frame. This is bad. Our fix will be a very simple one, we’ll just flip the order of drawing and ticking.

updateAnimationState = () => {
  const { engine } = this.state;
  if (engine) {
    // Draw the last frame we created
    engine.drawFrame();
    // Now let's make a new one!
    engine.tick();
  }
  requestAnimationFrame(this.updateAnimationState);
};

Now it doesn’t matter how long the tick lasts, because the actual drawing will be synchronized to the start of every frame! There are some additional things you could do to improve this. Ideally, your tick function is running on a separate thread via a web worker, and it would be great if you had some way to double buffer your images, though these things are beyond the scope of this post. The gist of it is that you always want to have something new to display on your animation frames, but you should have computed what to display beforehand.

If you’ve read this far, you may have felt that what we’re doing here flies in the face of React. We’re altering the state of a component and displaying new visuals to the user without telling the framework, a big no-no. React is meant for building user interfaces, and graphics like the Canvas API or WebGL just don’t fit into the framework. Rarely do we as developers ever compose just a single element in isolation. We weave together dozens of little elements to create a cohesive unit, and sometimes that means shoving things where they don’t fit so well. You’ve got a React app, and you want high-performance graphics. Should you have to bend your graphics stack to fit the React philosophy, even if it means delivering a less performant element? Absolutely not. Go out there and make something fun, something inspiring, something useful. Writing beautiful code is fun, but creating beautiful software is better.

Hopefully this helped you along! If you want to reach out with suggestions or corrections, feel free to email me.