#debugging

[techav]

On Recovering From Disaster

Just before disaster struck with my 68030 homebrew, I had a plan to fork the project and take it in two directions in parallel.

The first would be my minimal Multibasic setup which was running great and was something I would be proud to exhibit at VCFSW '25.

Since that was working well, I didn't want to do anything that might make it not ready for the show. So I had assembled a second main board that I could use with other peripheral cards to continue development. My plan was to rework the memory map to move ROM and I/O addresses to the upper half of the 32-bit address space, which would allow me to use a DRAM card for up to 16MB of contiguous memory starting at address 0 — a perfect setup for running a proper OS

And then I burned up my good mainboard.

I had a spare CPLD on-hand, and I was able to get another 40MHz 68EC040 for cheap. But the 4Mbit SRAM chips are expensive, and I thought that if I was going to be paying for more SRAM, I should get the 12ns parts that would allow me to run RAM with no wait states at 50MHz. Which would require adapter boards, and more pin headers, and would have just driven the cost up even more.

Paralyzed by indecision and the rising cost, I decided to switch gears. I already have the DRAM board I built years ago, I just needed to get it working. And if I'm doing that, I may as well do the work on reorganizing my memory map.

So that's what I did. I made a new branch on my newly cleaned up repository, and set to work. I updated the mainboard bus controller logic to remove the no-longer-needed SRAM control, adjust the ROM address to start at 0x8000,0000, and move the 6850 UART address to 0x8008,0000. This freed up the entire 2GB space between 0x0000,0000 and 0x7FFF,FFFF for the DRAM card to use for main memory (my bus connector layout only allows for up to 16MB main memory, but that's plenty for my purpose).

Next, I needed to build the control logic for my DRAM board. I've done this once before on my Wrap030-ATX project a couple years ago, so I used that logic as a starting point. It … kind-of worked. Sometimes. Maybe.

I was getting random errors all over the place. Sometimes it was able to copy all of ROM into RAM and verify the copy without error, but it would fail when running code from RAM. Sometimes it wouldn't even pass the first two simple tests of writing a longword to RAM and reading it back.

Back to the data sheets to plan out the logic. I drew out a new timing diagram down to 1ns resolution, accounting for the specified propagation time for my CPLDs, and the measured signal delays for my CPU. This gave me the information I needed to build out a new state machine for the DRAM controller. In the process I also caught a few other bugs in my original logic, such as not ensuring the DRAM Write Enable strobe did not get asserted during refresh cycles.

The new state machine worked much better, but I got my timing off a bit. It worked reliably at 16MHz, but not at the intended 25MHz. At least it was working though; I can move forward with 16MHz.

The next thing to do was get my 8-port serial card working with this new setup. Every time the computer tried to access one of the 8 serial ports, it would get a bus error. This was a tough one that had me stuck for a while.

It looked like the serial card was getting addressed and asserting DSACK0# as expected, but I couldn't confirm the signal was making it back to the CPU. Every time I tried to measure the DSACK signals, the probes would increase the line capacitance (I think) too much and cause the whole computer to fail before it even got to the point of trying to address the serial ports.

I have the DSACK signals in a wired-or configuration. The logic on each card connects to the proper DSACK signal via an open-collector buffer, and is able to pull the signal low to signal that card acknowledging the bus cycle. The signal is then pulled back up to +5V by a 1kΩ resistor. This works well enough for small, slow systems, but with long bus wires or lots of chips on the bus, the extra capacitance means it can take a long time for that pull-up resistor to get the signal back up to +5V.

I made a change to my DRAM controller to actively drive the DSACK signals to +5V briefly at the end of a bus cycle before setting the pin to high-impedance. This helps ensure a fast rise time at the end of the cycle so it doesn't disrupt the next bus cycle.

It didn't fix the problem with the serial card, but it did allow me to actual probe the signals live to see what was happening — the Bus Error signal was getting asserted immediately after the CPU addressed the serial card. This didn't make much sense to me. My main board logic will assert Bus Error if no peripherals acknowledge a bus transaction within 64 clock cycles. Nothing should be asserting Bus Error so quickly.

Except … the main board bus controller. My original memory map used the lower half of the address space for the main board, and peripheral cards used the upper half. So my original main board logic was built to assert Bus Error immediately for an unused address in the lower half of the address space. I hadn't properly removed that bit of logic, so it was now asserting Bus Error for any accesses to the upper half of the address space that weren't used by the main board.

The challenges of working on a project after six years. I had forgotten that bit of logic was even there.

With that erroneous Bus Error assertation removed, the machine was finally able to properly load my Multibasic kernel and run BASIC for 8 users again.

At the moment it is running stable at 24MHz on my 25MHz-rated full 68030. It may not be the 50MHz high I was at, but after the frustration and disappointment of the past few weeks, I'm absolutely taking that as a win. 24MHz is incredible.

[python-programming-language]

Different ways of debugging my source code ...

Post #88: www.pythonforbiologists.com, My code isn't working, 2023.

[noxiatoxia]

SDR2 Debugging Menu: Ongoing Mystery

On July 24th, 2017, a post was made to Reddit with the following screenshot:

The post title read, "This menu appeared in SDR2 when I pressed random buttons?"

Commenters were quick to point out this looked like a Debug menu, and asked OP how they had managed to open it.

Op had this to say:

"I was using a PS4 controller. I pressed Triangle,X, and both L1 and R1 at the same or close times in general, but I don't know the exact combo"

It has been 8 years since this post. To my knowledge, nobody since has talked about this, nor has found a way to replicate what u/Nargieo managed to pull off.

Let's talk about it.

Firstly, you may ask, is it a hoax?

Obviously, the chance is never 0, and I'll take the L if it is fake. But I personally believe it to be real.

The OP's account is suspended now, but using the Wayback machine, they seemed to be a fairly standard user who was a fan of multiple games and anime, Danganronpa just happening to be one of those things.

Furthermore, the font seen in the screenshot matches the debug_menu.ttf font file found inside the files of SDR2.

The words on the menu themselves also match how certain aspect are referred to in files only. For example, in the games files, Island Mode is called "SV" or "Survival Mode", matching the "Surv_Char" and such we see on that menu.

According to a friend of mine that knows more about game dev than I do, it is apparently common to use the controller of a "bigger" console (a PS4 opposed to a PSVita in this case) to play test games. A Debug menu being coded in specifically for this controller but not the Vita controller would be normal, they said.

Obviously, the chance of it being faked is possible. But considering all the errors and things left inside the Dangan games by mistake as it is, this seems apt. I believe it is real.

Why has no one talked about this in 8 years?

That's a good question. I don't know. But I feel it should be looked into more than it is now. Having access to the Debugging menu could give us more insight into the game than simply rifling through the files could. It is also just a cool thing to have.

By the way, considering how SDR2 is built, finding the debug menu for it likely means whatever button input will also work to open the debug menu for DR1.

What we know:

This took place on the PC version of SDR2. Based on the date being 2017, it is likely the newest Steam build which came out in Sept. 2, 2016. Build ID is 1312489.

Since the mouse icon is still used in the textbox to prompt the user to advance text, they had Keyboard and Mouse enabled for control hints. Not Gamepad, despite using a PS4 controller.

Speaking of, the user recalls pressing L1, R1, X, and △ in some order. This should be taken with a grain of salt, as they may be misremembering or have pressed something else without knowing. Just keep an open mind that other buttons may have been used.

Since we see a character (Kazuichi) speaking, we can assume it is unlikely (but not impossible) that the Options or □ button were pressed, as both these buttons open a new screen (the "Help" screen and Student Handbook respectively).

As the User said, they achieved this by pressing "random buttons" and were shocked to find this. Meaning something very important: This screen is something one could find on accident. That's important as it rules out any long codes that would be near impossible to input on accident, or use of the PS4 touch pad in the case of writing something specific.

The question is now: how do we replicate this?

That's the million dollar question. I, personally, have spent 3 hours so far in the middle of trials pressing random buttons and any combination of L1, R1, X, and △ that I can think of. I haven't had any luck so far.

I can't do this by myself, obviously. So I implore everyone who is interested (and has access to both SDR2 on Steam and a PS4 controller) to try this themselves. If you have any insight on Game Development, please add your thoughts.

However, accessing the debug menu without knowing how you did it is pointless. So here is what I suggest:

Open SDR2. Make sure it is not full screen. Have it off to the side.

Use the website Gamepad Viewer to visualize your PS4 controller.

Screen record as you play. I suggest OBS if you don't have a screen recorder, but anything works.

If you don't get the debug menu and you feel like you're done playing, just delete the video file. If you do get it, you can stop recording and look back at the footage to see what exact combo of buttons you pressed.

It may look something like this, set up wise:

If nothing else, I hope this post gains traction so knowledge of this re-enters the Danganronpa community. It would be a shame to leave this to fade into obscurity.

[hauxicrook]

[el-ffej]

Text below:

Angels: so what's up with the platypus

God: yeah, about that

Angels: seriously

God: idk it doesn't do anything but it I take it out the world won't compile

[code-tryperse]

Bugs in Sky: CotL: Broken Gates and Lonely Areas

Disclaimer: I am not affiliated in any way to TGC. Barring confirmation from the developers, everything below is a hypothesis based on observed game behavior.

From January 16, 04:02 GMT (the release of update 0.28.0) up until January 20, 04:03 GMT, Sky players experienced a bug where players rarely encountered other players, resulting in a rather lonely experience playing the game. At the same time, gates and other objects which required lighting an interactable element to activate didn't activate even after lighting said element. Most players attributed this bug to server merging (i.e.: a function where two servers have their players and state combined into one) not working as intended.

This wasn't the case, however, as it was still entirely possible to encounter other players at times. It was just happening a lot rarer than it used to. Instead, a related function — server splitting — was not working as intended which, in turn, caused server merging to happen extremely rarely.

Potential explanation of what happened below the cut.

TLDR: The 0.28.0 update to the game likely accidentally disabled server splitting in all areas of the game, causing server merging to rarely happen and for some gates to stop working.

Keeping Players Close: Server Splits and Server Merges

Have you ever had an experience where you were with your friends when all of a sudden, they all disappear in front of you? And when you teleport back to them, they apparently were still in the same place as they were? Congratulations, you just encountered a server split. A server split is where a player or group of players get disconnected from a server they're currently in and pushed into their own server. Sometimes, the server will separate everyone out of it in what will be called here a "full server split". In that case, everyone gets pushed into a server where they're the only player. The scenario mentioned above is a very noticeable example of a full server split happening. Most of the time, however, server splits don't happen to every player in the server. Instead, it only happens to one player or group of players in that server. This kind of separation is what this blog will refer to when mentioning the term "server split".

The exact criteria needed for a server to split a player or group of players isn't known. However, one criterion that can be easily observed is that the server will split a player or group of players when there are no other players within close proximity to them. "Close proximity" is not defined in strict terms here as it is currently not known how far this distance needs to be. However, to observe this behavior, you can do the following steps:

Stand on one of the floating islands in Sanctuary Island.

Have a friend fly towards the main island.

Keep honking in order to see your friend's name.

As they fly down to the main island, eventually, your friend's name will stop appearing when you honk. A server split just happened. Because you are far from your friend, the server thinks that it is alright to split the two of you in order for a server merge to bring either of you to a server where players are close to you or your friend.

This is necessary behavior because there's a limit of 8 players per server. In order for a server merge to be successful, there must be at most 8 players in both servers combined. By splitting players from the server they're in, it increases the chances that the game will find a server that they can be merged in by reducing the number of players in their server. This is why waiting inside the 8-player puzzle room when there's already 7 players there usually takes a long time. By having 7 players in the server, only a server with one player can be merged into it. However, the two-player door before the room means that most servers with players in that room will have at least two players. Those servers cannot be merged into your 7-player server.

Determining the Absence of Server Splits

To say that server splitting isn't working as intended, we can establish the existence of players that should have been split away from us within the server. Normally, it would be hard to determine such a case, as that entails having to search for the presence of players in areas we cannot see. Additionally, if we do spot a player, we cannot know if that's a player who was already there, or one who was just merged into the server. Thankfully, there is a way to determine the presence of players without seeing them: the chat history.

When you send a chat message in Sky, it is sent to all players in the server you are in, including yourself. This prompts the game to display that chat message on top of the player who said it as well as in the chat history. However, if you don't have chat unlocked with the player, or if either of you aren't in a chat bench (doesn't have to be the same one), the game will replace the text of the message with a bunch of asterisks when displaying it to you. Additionally, if you have a player muted, the game will not display the messages they are sending, but it will still store them. This can be verified by muting a player, having them say something, then unmuting them. The game will then show the messages they sent while you have them muted, which means that it stores their messages even when they're muted.

Because player messages are sent to the entire server, you can see the messages of players in other locations, provided that they are in the same server as you. You can verify this by joining a friend (chat can be locked or unlocked), going into a level transition, then having them chat something. Their messages will still appear in the chat history until the server splits you off from them.

We know that this happened during the period of the bug because there are numerous bug reports in the Sky Discord mentioning the appearance of messages from players who are nowhere near the reporter. These messages can only appear if a player had sent them, which means that there's a player somewhere in that server sending these messages. This player is far from the reporter, which means that they should have been split off from them. However, that is not the case, which shows that server splitting is not working as intended.

Gates and Their Dependencies on Server Merging

Another part of the bug is that none of the gates and other places which require lighting some sort of pedestal or key to open work. While the reason for this cannot be definitively determined, it is possible that these required a check that involved a server merge for it to start working properly. Given that a server merge usually happens often and sometimes unnoticeable, it is likely that a server merge was made as a requirement to ensure that some gates only work when there's multiple players in a given area. The presence of this requirement doesn't make much sense, as two-player and four-player doors already require multiple players on their own, and requiring multiple players for one-player doors is not intuitive and sometimes even harmful for user experience. However, this is the most likely explanation as to why this bug is related to the server splits bug.

Reworking Maps and Inadvertent Changes

To give a possible explanation as to how this bug happened, we need to look at a handful of places and scenarios, namely:

Homespace and Aviary Village

Eye of Eden and Orbit

Valley of Triumph Races and Red Shard Memory Collections

Homespace and Aviary Village serve as central hubs for players to go through. As such, it is important that players be capable of gathering in these areas. Server splits would require players to not explore the central hub while waiting for others, which can make for a boring experience for some. Additionally, full server splits may also happen, which will force the players to repeat the process of gathering, which can be annoying to deal with. As such, server splitting is disabled in these areas.

Past the first area of Eye of Eden, server merging is disabled. While the game's tip for Eye of Eden is to stick together, keeping server splitting enabled is too punishing, as it prevents a player straying too far ahead or too far behind from receiving help from other players. Additionally for Orbit, keeping server splitting enabled can prevent players from hanging around with the people they shared the experience of the end with, which can reduce player satisfaction. As such, server splitting is disabled in these areas.

Races in Valley of Triumph as well as memory collection in Red Shard events sum up all the light/memories that all players in a server receive. If a player within a race or within the memory collection activity were to get split off from the server, it could make the light calculation inconsistent. To prevent these, server splitting is disabled while a race or any other activity is active.

What this all shows is that server splitting is something that can be enabled or disabled when needed. In preparation for the new season, nearly all areas of the map have been updated for the new dye light mechanic. It is possible that, while updating the maps, server splitting was accidentally disabled in these areas, causing server splitting to be disabled in all areas of the game.

A Quick Hotfix

An update was released on January 20, 04:03 GMT which fixed this issue completely. Server splitting now works as intended again. This didn't require a manual update of the game, which meant that this was a fix that involved only editing the assets of the game and not the actual code. Either way, players can now continue playing the game, finding others who are also playing near their character.

[adafruit]

A little Espressif helper for programming

While developing boards, we often want to program ESP chips without going through the onboard USB port; this adapter will help us (and others) do that! It has a CP2102N USB-serial chip

https://www.digikey.com/short/bm7n3p5z

with RX/TX signal LEDs and two transistors wired up to the DTR/RTS line for the 'esptool standard' reset procedure technique. The output IO, plus a 3.3V 500mA regulated output, is available on a socket header, so you can plug wires in for quick programming and debugging. You can use this for everything from an ESP8266 to an ESP32-P4!

[thehattyguy]

Been deveveloping the world system for my game and ended up separating physics and logic from the rendering aspect of the game

That made it a bit hard to debug collisions and things related to the "logic" world because its all hidden within their own World3D so that I can have multiple worlds that don't interact with each other

So I made this small modular debug menu and added the first debug option which creates a duplicate camera inside the main logic world (maybe I'll make it move to the current focused world when I make ways of travelling between worlds) and overlays the rendered image on top of the regular game

This already helped me fix some bugs and there are still some others that showed up since I split logic and rendering that I'm still working on

Also! I had made the equipment section of the inventory binder and didn't end up making a post about it but it came out pretty nice aswell, I'll maybe post some more details about it when its more finished since for now you can only see and edit your deck

Here's how it's looking right now:

[peachgull]

Debug scraps I found at :

[askagamedev]

Debugging Complications

Sometimes a bug is easy to catch. You do some normal set of inputs and the aberrant behavior appears. Sometimes a bug is difficult to reproduce. Here are some of the qualities that can make a bug difficult to nail down:

[Build-specific] - The bug only shows up in release or final builds.

Platform-specific - The bug only shows up on a particular platform (e.g. PS4, Linux dedicated server, Windows 10 service pack 2, etc.)

Random timing - The repro steps are always the same, but the bug doesn't always happen when the steps are taken

Frame-specific timing - Specific inputs must be made within a specific frame window to cause the bad behavior. Bonus difficulty if you have to reproduce the bug on a debug build with a low frame rate

Silent failure - The bug doesn't actually trigger any error messages or logs or anything, it just fails silently without a trace

Requires multiple players to repro - Happens only on a dedicated server with multiple squads all playing at once! Player A must do the action to Player B, and the bug appears on player C's screen!

Requires a long time to reproduce - The bad behavior happens only when leaving the game on for multiple hours, like via a [smoke test]

Live Environment Only - This bug only ever shows up in the live game

Even more fun, there's nothing stopping more than one of these complications to happen within the same bug. A build-specific, platform-specific, live-only bug that silently fails and requires a bunch of players that only happens after many hours? Fun!

[Join us on Discord] and/or [Support us on Patreon]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

Frequent Questions: The FAQ

[omeletcat]

I know when a tutorial or like person who has been programming for a long time says "you should take errors as tips and a helping hand instead of something negative" they try and help you out! and i bet that would make error hunting way less annoying but like.. no, i hate errors, if it was a person i'd punch it in the face.

i'm sorry errors, i'm sorry debug menu, but we can never be friends.

[theftmprogrammer]

ever have to debug something so frustrating you have to go touch grass?

[dogeiscut]

WHAT THE FUCK

I GOT THE FUCKING QUAD BLUESCREEN AND SUMMONED MISSINGNO???? WHY DOES IT LOOK LIKE THAT

I would like to note all I was doing was coding and my internet went out on the pc right before it froze and crashed like this

[zemmouriaymen]

Every programmer has that one piece of code… The one that made sense at 3 AM but now looks like ancient hieroglyphics. 🫠

💻 “I knew how it worked… now, only heaven knows.”

🔧 total_hours_wasted_here = 254

Who else has written code like this? Tag your fellow suffering devs. 😂

[transmhyra]

stalin style debugging