Differences

This shows you the differences between two versions of the page.

--- flag_operations_are_free [2026/01/12 02:26] – appledog
+++ flag_operations_are_free [2026/01/15 05:32] (current) – appledog
@@ Line 130: / Line 130: @@
 </codify>
-for more information please contact Appledog.
+== The Prefetch System
+Combining the loads and bit-shifting things out worked so well I had the brilliant idea to create a prefetch system. Why not batch loads across instructions? This would allow us to capitalize on aligned loads but still have long instructions (5 bytes and up) without running into alignment problems.
+Here's the code, preserved for posterity, may it serve a lesson to all that too much of a good thing actually sucks:
+<codify ts>
+// ============================================================================
+// INSTRUCTION PREFETCH BUFFER SYSTEM
+// ============================================================================
+//
+// Purpose: Reduce memory operations by batching instruction fetches.
+// Instead of loading 1 byte at a time from RAM, we load 4 bytes at once
+// and extract individual bytes from the buffer as needed.
+//
+// Trade-off: Adds overhead from buffer management, but amortizes memory
+// loads across multiple instruction bytes.
+//
+// Performance note: This system was intended to improve performance by
+// reducing memory operations, but in practice added overhead that reduced
+// MIPS from ~90 to ~60. WebAssembly's memory access and CPU prefetching
+// are already well-optimized for sequential loads.
+// ============================================================================
+// Prefetch buffer holds up to 4 bytes of instruction stream
+let prefetch_buffer: u32 = 0;        // The 4-byte buffer holding fetched data
+let prefetch_valid: u8 = 0;          // How many valid bytes are in buffer (0-4)
+let prefetch_offset: u8 = 0;         // Current read position within buffer (0-3)
+/**
+ * Fetch a single byte from the instruction prefetch buffer.
+ *
+ * Automatically refills the buffer when exhausted by loading 4 new bytes
+ * from RAM at the current instruction pointer (_IP).
+ *
+ * @returns The next instruction byte
+ */
+function prefetch_byte(): u8 {
+    // Check if buffer needs refilling
+    if (prefetch_offset >= prefetch_valid) {
+        // Load 4 new bytes from instruction stream
+        prefetch_buffer = load<u32>(RAM + _IP);
+        prefetch_valid = 4;
+        prefetch_offset = 0;
+        _IP += 4;  // Advance physical IP by 4 bytes
+    }
+    // Extract byte at current offset (0, 8, 16, or 24 bits from right)
+    let byte = (prefetch_buffer >> (prefetch_offset * 8)) as u8;
+    prefetch_offset++;
+    return byte;
+}
+</codify>
+You can imagine the rest of the code -- this is enough to understand the problem:
+    if (prefetch_offset >= prefetch_valid) is very bad.
+Let me put it this way. If I run an IF on every opcode, it slows the program by 50%, turning a 90 mips LDA benchmark into a 45 MIPS benchmark. Now, I tried a lot of different methods to try and get a better score than just using load<u8> when needed. The closest I got was a branchless <u64> prefetch version of the above, which got around 84 MIPS. Even just trying to load everything at the top of the loop (a u32 load) and bit-roating out the operands was no more than a 1% improvement (i.e. not worth the trouble, frankly).
+In fact, the way I got to a 95 MIPS benchmark for unrolled LDA operations was to simply use fetch_byte(). If I did anything else, //including inliling the load<> operations//, the program would run slower.
+At first glance you may wonder what on earth is happening. How did I beat the 87.5 MIPS implementation above? Simple, by not trying to cheat the system. As it turns out, load<> is already as optimized as it is going to get in Web Assembly. The epiphany is, we're running a simulation. And the host computer is actually doing a good job of helping us access memory. Any abstraction we put on top ends up getting in the way.
+It's strange but true. If you factor out the load operations, trying to load everything at once adds complexity:
+    let addr = (instruction >> 8) & 0x00FFFFFF;      // Extract upper 3 bytes
+You're adding a bit shift, a bitwise AND, plus you're creating an intermediary variable access. In the end this is almost 10% slower than just calling fetch_byte().
+== Conclusion
+The grass is green, the sky is blue.
+<codify ts>
+export function cpu_step(): void {
+    let IP_now:u32 = _IP;
+    // Pre-fetch 4 bytes but only commit to using opcode initially
+    let opcode = fetch_byte();
+    switch(opcode) {
+        ///////////////////////////////////////////////////////////////////////
+        // LDR/STR load and store architexture                               //
+        ///////////////////////////////////////////////////////////////////////
+        case OP.LD_IMM: {
+            let reg = fetch_byte();
+            // Determine width based on register number
+            if (reg < 16) {
+                // 16-bit register
+                let value:u16 = fetch_word();
+                set_register(reg, value);
+...
+</codify>
+Nothing beats this. This is it. You can't even inline it, it messes with the compiler.
+If you want to get faster than this, you need to rewrite the entire switch in C or maybe Rust.
+As a result of all this, I now know that flag operations are free inside an opcode and I don't need to have a "fast flags" bit. Simply checking that bit every instruction was slowing down the system by far more than it saved.
+//Moral: "Premature optimization is the root of all evil."//