Go Down

Topic: ROM-Reader for Super Nintendo / Super Famicom Game Cartridges (Read 80654 times) previous topic - next topic

skaman

Got around to redumping Star Ocean.  With the Adafruit Clock Generator connected, Star Ocean cart no longer needs the verifySORead code.  The SDD1 carts now dump correctly every time.

EDIT:  Ran tests without the clock and Star Ocean still dumped okay.  Maybe it isn't the Clock Generator?  I did a couple runs and both dumps were good without the verifySORead.  On the other hand, the clock is definitely needed for FEOEZ.

skaman

Made a huge time savings in calc_checksum().  Added a 512 byte buffer for the myFile.read() and it now completes the task in seconds versus minutes.  I'll post the code after I clean it up.

EDIT:  Applied the same 512 byte buffer to myFile.write() in dumpROM() and it cut my dump times by more than half!  Got a 2MB (16Mbit) LoROM dumping AND checking the checksum in 1 minute - it used to take around ~3:50. 

In case anyone cares, the 512 byte buffer was chosen as it is the minimum block size used by the SD card.  Reading and writing anything less than 512 bytes to the SD card slows things down.

sanni

I've confirmed all of the SDD1, SuperFX, SPC7110 and Seta DSP (ST010/ST011/ST018) carts dump properly.  I also tested one of the CX4 carts - Rockman/Mega Man X2.  The other CX4 cart - Rockman/Mega Man X3 should work but it is untested.  
This is so amazing  :)
It perfectly shows the power of open source  8)



skaman

Okay, here's the SD buffer code to speed up reads and writes.  I'll post the complete dump code after I test the ExHiROM carts and a couple other enhanced chip carts.

First off we add the SDBuffer[512] in the define section.

Next, my current calc_checksum() code.  Some of it is untested as I don't have all of the different sizes (missing 1.5MB, 2.5MB & "Normal" 3MB).  I referred to ROMs from the GoodSets where needed.

Code: [Select]

//Define Cart Reader Variables
...
byte SDBuffer[512];
...

//*************************CALCULATE CHECKSUM**************************************
unsigned int calc_checksum (char* fileName, char* folder) {
  unsigned int calcChecksum = 0;
  unsigned int calcChecksumChunk = 0;
  int calcFilesize = 0;
  unsigned int c = 0;
  unsigned long i = 0;
  unsigned long j = 0;

  if (strcmp(folder, "root") != 0)
    sd.chdir(folder);

  // If file exists
  if (myFile.open(fileName, O_READ)) {
    calcFilesize = myFile.fileSize() * 8 / 1024 / 1024;
    if (calcFilesize == 12 || calcFilesize == 20) {
      // Divide filesize by 8 to get number of 8Mbit chunks
      for (i = 0; i < (calcFilesize / 8); i++ ) {
        // Add all the bits in the 8Mbit chunks
        for (j = 0; j < (1048576 / 512); j++) {
          myFile.read(SDBuffer, 512);
          for (c = 0; c < 512; c++) {
            calcChecksumChunk += SDBuffer[c];
          }
        }
        calcChecksum = calcChecksumChunk;
      }
      calcChecksumChunk = 0;
      // Add the 4Mbit rest
      for (j = 0; j < (524288 / 512); j++) {
        myFile.read(SDBuffer, 512);
        for (c = 0; c < 512; c++) {
          calcChecksumChunk += SDBuffer[c];
        }
      }
      calcChecksum +=  2 * calcChecksumChunk;
    }
    else if (calcFilesize == 24 || calcFilesize == 28) {
      // Momotarou Dentestu Happy Fix 3MB (24Mbit)
      if ((calcFilesize == 24) && (romSizeExp = 0x0C)) {
        for (i = 0; i < (myFile.fileSize() / 512); i++) {
          myFile.read(SDBuffer, 512);
          for (c = 0; c < 512; c++) {
            calcChecksumChunk += SDBuffer[c];
          }
        }
        calcChecksum = 2 * calcChecksumChunk;
      }
      else {
        for (i = 0; i < (calcFilesize / 16); i++ ) {
          // Add all the bits in the 16Mbit chunks
          for (j = 0; j < (2097152 / 512); j++) {
            myFile.read(SDBuffer, 512);
            for (c = 0; c < 512; c++) {
              calcChecksumChunk += SDBuffer[c];
            }
          }
          calcChecksum = calcChecksumChunk;
        }
        calcChecksumChunk = 0;
        // Add the 8Mbit rest
        for (j = 0; j < (1048576 / 512); j++) {
          myFile.read(SDBuffer, 512);
          for (c = 0; c < 512; c++) {
            calcChecksumChunk += SDBuffer[c];
          }
        }
        calcChecksum +=  2 * calcChecksumChunk;
      }
    }
    else if (calcFilesize == 48) {
 // Star Ocean/Tales of Phantasia Fix 6MB (48Mbit)
 // Add the 4MB (32Mbit) start
      for (j = 0; j < (4194304 / 512); j++) {
        myFile.read(SDBuffer, 512);
        for (c = 0; c < 512; c++) {
          calcChecksumChunk += SDBuffer[c];
        }
        calcChecksum = calcChecksumChunk;
      }
      calcChecksumChunk = 0;
      // Add the 2MB (16Mbit) end
      for (j = 0; j < (2097152 / 512); j++) {
        myFile.read(SDBuffer, 512);
        for (c = 0; c < 512; c++) {
          calcChecksumChunk += SDBuffer[c];
        }
      }
      calcChecksum +=  2 * calcChecksumChunk;
    }
    else {
    //calcFilesize == 2, 4, 8, 16, 32, 40, etc
      for (i = 0; i < (myFile.fileSize() / 512); i++) {
        myFile.read(SDBuffer, 512);
        for (c = 0; c < 512; c++) {
          calcChecksumChunk += SDBuffer[c];
        }
        calcChecksum = calcChecksumChunk;
      }
    }
    myFile.close();
    sd.chdir();
    return (calcChecksum);
  }
  // Else show error
  else {
    display.println("ERROR");
    display.println("DUMP ROM 1ST");
    display.display();
    return 0;
  }
}


The checksum calculation greatly benefits from the 512 byte buffer.  We read the file on the SD card in 512 byte chunks to perform our calculation.  Calculations used to take minutes but now it takes only seconds to complete.

Now for the dumping code.  In an effort to streamline the dumpROM() code, I created a dumpLoROM() and dumpHiROM().  These routines dump 512 bytes into the SDBuffer then writes the buffer to the file on the SD card.

Code: [Select]

long dumpLoROM() {
  display.println("Dumping LoROM");
  display.print("Banks: ");
  display.println(numBanks);
  display.display();
  long dumpedBytes = 0;
  // Read up to 96 banks starting at bank 0x00.
  for (int currBank = 0; currBank < numBanks; currBank++) {
    // give status updates via LCD
    display.print(".");
    display.display();

    // Dump the bytes to SD
    for (long currByte = 0x8000; currByte < 0x10000; currByte += 512) {
      for (unsigned long c = 0; c < 512; c++) {
        SDBuffer[c] = dumpByte(currBank, currByte + c, false);
        dumpedBytes++;
      }
      myFile.write(SDBuffer, 512);
    }
  }
  return(dumpedBytes);
}

long dumpHiROM(int currBank, int endBank, long dumpedBytes) {
  for(currBank; currBank < endBank; currBank++) {
    // give status updates via LCD
    display.print(".");
    display.display();
   
    // Dump the bytes to SD
   for(long currByte=0; currByte < 0x10000; currByte += 512) {
      for (unsigned long c = 0; c < 512; c++) {
        SDBuffer[c] = dumpByte(currBank, currByte + c, false);
        dumpedBytes++;
      }
      myFile.write(SDBuffer, 512);
   }
  }
  return(dumpedBytes);
}


So substitute dumpLoROM() and dumpHiROM() into dumpROM() where needed.  Here's the calls as I have them right now.

Code: [Select]

void dumpROM() {
...
  long dumpedBytes = 0;
  ...
  if ((romChips != 67)&&(romChips != 69)&&(romChips != 245)&&(romChips != 249)) {
    if (romType == 0) {
     ...
       dumpedBytes = dumpLoROM();
    ...
    }
    else if (romType == 1) {
      ...
        dumpedBytes = dumpHiROM(0xC0, numBanks + 0xC0, dumpedBytes);
    ...
    }
  }
  else if((romChips == 67)||(romChips ==69)) {
    // SDD1
    // Street Fighter Alpha/Zero 2 (J, U, E): romChips = 67
    // Star Ocean (J): romChips = 69
    ...
      dumpedBytes = dumpHiROM(0xF0, 0x100, dumpedBytes);
  ...
  }
  else if((romChips == 245)||(romChips == 249)){
    if (romType == 0) {
      // ST018
      // Hayazashi Nidan Morita Shougi 2: romChips = 245
      dumpedBytes = dumpLoROM();
    }
    else if (romType == 1) {
      // SPC7110
      // Momotarou Dentetsu Happy/Super Power League 4: romChips = 245
      // Far East of Eden Zero: romChips = 249
    ...
      dumpedBytes = dumpHiROM(0xC0, 0xE0, dumpedBytes);
    ...
      dumpedBytes = dumpHiROM(0xE0, 0xF0, dumpedBytes);
    ...         
      dumpedBytes = dumpHiROM(0xF0, 0x100, dumpedBytes);
    ...
      dumpedBytes = dumpHiROM(0xF0, 0x100, dumpedBytes);
    ...
    }
  }
}

skaman

Got a code change for the CX4 (Rockman/Mega Man X2 & X3) carts.  I had previously only tested the code on X2 but I was finally able to test on X3.

Rockman/Mega Man X2 & X3 both start with the mapping register 0x7F52 set to 1.  X2 needs to be set to 0 in order to fully dump the ROM.  X3 needs to be set to 1 to dump the entire ROM.

I added code to differentiate between X2 and X3 and also switched the register setting code to use the slower setSRAMByte() to make sure the changes worked. 

Code changes:

Code: [Select]

void dumpROM() {
...
    if (romType == 0) {
      byte initialCX4Map = 0;
      byte mmxType = 0;
      // CX4 (Mega Man X2/Mega Man X3/Rockman X2/Rockman X3) romChips = 243 (0xF3)
      if(romChips == 243){
        // Check initial content of mapping register...
        // Mega Man/Rockman X2 & X3 should both return 1
        initialCX4Map = dumpByte(0, 0x7F52, false); //0x7F52
       
        display.print("CX4 Register: ");
        display.println(initialCX4Map);
        display.display();

        // Check ROM Name to determine whether X2 or X3       
        mmxType = dumpByte(0, 0xFFC9, false) & 0xF; // 0xFFC9
     
        // Mega Man/Rockman X2 needs to be set to 0
        if((mmxType == 2)&&(initialCX4Map != 0)){
          setDataOut();
          // Set 0x7F52 to 0
          setSRAMByte(0, 0x7F52, 0, false);
          setDataIn();
        }
        // Mega Man/Rockman X3 needs to be set to 1
        if((mmxType == 3)&&(initialCX4Map == 0)){
          setDataOut();
          // Set 0x7F52 to 1
          setSRAMByte(0, 0x7F52, 1, false);
          setDataIn();
        }
      }
      dumpedBytes = dumpLoROM();
      if(romChips == 243){
        // Return mapping register to initial setting...
        setDataOut();
        setSRAMByte(0, 0x7F52, initialCX4Map, false);
        setDataIn();
      }
    }
...
}

Gargamel13

Is the latest version of this source made available public?  I don't want to make a mistake pasting pieces of code from the forums together... :smiley-confuse:

skaman

My code is in constant flux so I haven't posted the complete code.  I've been testing the special enhanced chip games.  So far, I've dumped all of the CX4, OBC1, SDD1, S-RTC, ST010, ST011, ST018, and SuperFX carts.  ExHiROM carts also work but it requires byte verification code like verifySORead.

I'm finishing up testing on the ExHiROM and S-RTC (Daikaijuu Monogatari 2) code.  I'll post my complete code after I test a couple other carts that should be arriving soon.  I also have to fix a bug that prevents writing to the SD card (any cart that uses an extended ASCII character in the ROM name will not be able to write to the SD).

If you're building the reader, then the original codes that have been posted will get you up and running.  There are only a small number of enhanced chip games.

skaman

Some progress notes...

S-RTC, DSP1, DSP2, and DSP3 all dump, read/write SRAM.  I only tested one DSP1 cart (Super Mario Kart) but it works.  Waiting for a DSP4 cart to get here. 

The S-RTC cart has been kicking my @ss trying to get the RTC data.  There isn't much documentation on the S-RTC chip so communicating with it is like flying blind.  I've got the chip changing modes but the read command isn't working for me right now.  Too bad the chip isn't fully documented like the EPSON RTC used in FEOEZ which has the full datasheet available.

I also added code to dump the Nintendo Power (SF Memory Cassette) cart and I'm trying to see if the SRAM is accessible without any special commands.

skaman

Breakthrough!

I managed to dump the SA-1 cart.  I actually dumped PGA Tour 96 yesterday before my previous post.  I was going to include the news in that post but since I only had only managed it once (I finished the hardware addition that same day), I deleted the details before posting.  I managed to dump the same cart again today and verified the contents of the ROM which appears to be a previously undumped version of the game.  My ROM and SRAM dump both work in higan although I haven't played thru a game yet.

Dumping the SA-1 cart requires attaching a CIC chip in lock configuration.  I built two different versions of the CIC lock - one using a SuperCIC 16F630 and one with a snesCIC 12F629.  I've dumped the PGA Tour 96 using both versions.  The cart connector CIC pins (24, 25, 55, 56) need to be connected to the CIC lock chip.  Pin 56 is connected to the CIC chip and a 3.072MHz clock signal (I'm using one of the other clock outputs from the Adafruit Clock Generator).

The CIC-SA1 connection is a bit hit-or-miss right now.  Most of the time it doesn't connect but it will eventually work.  I think I have to tweak the reset timings to get it working more consistently.  What was amazing is that I managed to dump Super Mario RPG on the first attempt using the SuperCIC.  It was nice to finally get a copy of my savegame off the cart.  Kirby Super Star dumped on the first attempt with the snesCIC.  Kirby's Dream Land 3 dumped ok but is giving inconsistent SRAM dumps so I suspect the battery might be dying.

Another good thing about the CIC addition is that it works fine with the other non-SA1 carts.  There is one cart that does need to have the CIC module removed and it is the Nintendo Power (SF Memory Cassette).  With the CIC module connected, the Nintendo Power cart won't be recognized by the reader.  I'm assuming the CIC module and the MX15001TFC processor chip don't get along.  I'll have to do more testing to see if there is a workaround.

EDIT:  The changes to the setup() code with the CIC Lock Reset outlined below have fixed the issue with the one problematic Nintendo Power cart.

I'll post more details after further testing.


tylau0

Excellent! Thanks for sharing the big news.

I am now looking forward to receiving my electronic parts so I can try it out here.

MichlK

I managed to dump the SA-1 cart.
Wow - congratulations once again! :o There are not many solutions out there that support all the enhancement chips... :)

The CIC-SA1 connection is a bit hit-or-miss right now.  Most of the time it doesn't connect but it will eventually work.  I think I have to tweak the reset timings to get it working more consistently.
I guess, you have already seen this, but there was some discussion in the nesdev.com forums about this - just search for "ripping sa1 nesdev" in Google...

Best regards
Michael

skaman

Some notes on the CIC stuff.

The most compatible version of the lock seems to be the snesCIC.  I've been getting the best results using the snescic-lock-resync version.  Get the ASM & HEX here:  https://github.com/mrehkopf/sd2snes/blob/develop/cic/

Code: [Select]

;   snesCIC LOCK 12F629
;   pin configuration: (cartridge slot pin) [original 18-pin SMD lock CIC pin]
;
;                       ,---_---.
;      +5V (27,58) [18] |1     8| GND (5,36) [9]
;      CIC clk (56) [7] |2     7| CIC data i/o 0 (55) [1]
;   host reset out [10] |3     6| CIC data i/o 1 (24) [2]
; CIC lock reset in [8] |4     5| CIC slave reset out (25) [11]
;                       `-------'



You wire up the snesCIC to the Cart Pins 24, 25, 55, 56.  Connect CIC Pin 2 and Cart Pin 56 both to the clock generator outputting 3.072MHz.  I also connected Host Reset (CIC Pin 3) and CIC Lock Reset (CIC Pin 4) to available pins on the Mega 2560.

The snesCIC has specific timing built into the code for the CIC LOCK-KEY to work.  We need to set up our code to make sure the LOCK-KEY works right.  Since the snesCIC has the correct timing, we need to trigger the code at the right time using the CIC Lock Reset.  When CIC Lock Reset is HIGH then it is in reset, when LOW it is running the CIC code. 

In our Arduino setup(), we immediately set CIC Lock Reset HIGH otherwise the LOCK-KEY communication may fail.  Pulling CIC Lock Reset HIGH gives us time to get everything else that is connected to the Arduino up and running before trying the CIC handshake.  We let most everything initialize in setup() then we set CIC Lock Reset LOW to start the CIC code and hopefully get a valid connection to the cart.

After making the changes to setup(), the SA-1 carts get recognized most of the time now.  For when the cart is not recognized, I added a soft_Reset() "jmp 0" to reset the reader and start over.  I've dumped all of my SA-1 carts except for one problem cart, PGA European Tour.  I don't know what is different about the cart but it only spits out 0x17s right now.

EDIT:  I managed to get the problematic PGA European Tour to dump. I put the cart in the SNES and let it run for a little bit then moved it over to the reader and dumped it.  I'm still not sure what makes this cart fail.  I'll have to look at the SA-1 registers to see what is there.

skaman

Success!

I finally fixed the S-RTC communication and have the S-RTC outputting the time correctly.  So both the RTC and S-RTC now output properly.  If anyone thinks it would be worthwhile to add writing the time to the RTC chips, then let me know.  I'm not inclined to add it as the time can always be set in the respective cart.

I've made progress on a different cart.  I managed to get the games off the Nintendo Power (SF Memory Cassette) cart in multi-game configuration.  It requires some fiddling with the MX15001TFC registers 0x2400 and 0x2401.

In single game configuration (with or without the Menu), the reader should be able to see the game data in banks 00-7F in LoROM (0x8000-0xFFFF).  After dumping the Menu, the game data will be at 0x80000 in the dump.  If you set the registers to switch to the game, then the data changes and the cart actually becomes the individual game and the normal cart dumping process will work.

I have a multi-game (3-in-1) cart and the games are not visible to the reader.  Only the Menu is visible and it repeats 8x (in banks 00-7F).  I searched all of the banks and there is no game data to be found.  The MX15001TFC hides the actual game data.  Accessing the game data requires setting registers 2400 and 2401, the data changes for each game so it requires multiple steps to dump all of the games.

Right now I can dump individual game data with the settings hardcoded but it will take some time to code it so that the reader can dynamically dump the games.  I have to wrap my head around coding all of this because of the number of steps involved. We need to access the Menu, read the game entries, set the correct registers, dump the individual game data, then reset the cart to repeat the process to dump the next game. 

I have to experiment more with switching between games.  Registers 2400 and 2401 remain active even when acting as a single game so I'm hoping it is possible to switch between the games with a reset instead going back to the Menu.

skaman

Ok.  So I appear to have sorted out the last inconsistency issue with the SA-1 carts.  The key is getting the clock signals exactly right in order to have the cart recognized and dump properly.  One of my SA-1 carts, PGA European Tour, almost never got recognized due to a slightly incorrect clock frequency.  I completely forgot that I had an old PIC 16F84 frequency counter so I never bothered to check the clock signals until yesterday.

Using the Si5351 Arduino library (https://github.com/etherkit/Si5351Arduino),  there is a calibration needed to get the clock output as close as possible to the target 21.477272 MHz and 3.072 MHz.  If you read the documentation on the library, then you will find the calibration section in the Readme.  You need to run the si5351calibration sketch in the Arduino IDE to find the setting for your individual setup.

After testing my clock generator with a frequency counter, I found that my clocks were at 21.46998 MHz and 3.07096 MHz.  So the clock generator definitely needed calibration.  I ran the si5351calibration sketch in the IDE and found the correction needed for my reader.  The set_correction value needs to be set early in the setup.  Here's how my clock generator section in setup() looks right now:

Code: [Select]

void setup() {
...
  // Adafruit Clock Generator
  clockgen.set_correction(-29000); // Find this value by using the si5351calibration in the Arduino IDE
  clockgen.init(SI5351_CRYSTAL_LOAD_8PF, 0);
  clockgen.set_pll(SI5351_PLL_FIXED, SI5351_PLLA);
  clockgen.set_pll(SI5351_PLL_FIXED, SI5351_PLLB);
  // Set Clock Generator CLK0 to 21.477272MHz
  clockgen.set_freq(2147727200ULL, SI5351_PLL_FIXED, SI5351_CLK0);
  // Set Clock Generator CLK2 to 3.072MHz
  clockgen.set_freq(307200000ULL, SI5351_PLL_FIXED, SI5351_CLK2);
  clockgen.output_enable(SI5351_CLK0, 1);
  clockgen.output_enable(SI5351_CLK1, 0);
  clockgen.output_enable(SI5351_CLK2, 1);
...
}



The clocks are now almost spot on at 21.47727 MHz and 3.07199 MHz.  With the current clock settings, the SA-1 carts are recognized every time even the problem PGA European Tour cart.  I'll be picking up a new frequency counter to replace my old PIC16F84 unit so maybe I can get the clocks exactly right.

For anyone planning on adding SA-1 support to their reader, the snescic-lock-resync is definitely the best version to use.  Don't bother with the SuperCIC as it doesn't recognize the SA-1 carts as reliably as the snesCIC resync version.

Using my reader, I dumped a new version of an SA-1 cart, PGA Tour 96 (U).  The cart that I dumped is Revision 1.1 with ROM chip "SNS-A3GE-1".  The previous known version is Revision 1.0 with chip "SNS-A3GE-0" (I dumped a copy of that one too).  I submitted the new dump to No-Intro and GoodTools so it will eventually be available for anyone that cares about this type of stuff.

skaman

Nice work, sanni!

I'll have to post a complete version of my reader code.  The reader code should support all of the enhanced chip carts.  I was hoping to include full support for the Nintendo Power (SF Memory Cassette) carts but the NP carts are finicky (even more than the SA-1 carts).  If I can't manage to get the NP cart working 100%, then I'll post a version with everything but the NP code. 

The biggest problem I'm having with the NP carts is I cannot consistently get the cart to switch to a single game.  Switching to a single game is important as that is the only way to see and dump the SRAM.  Once the cart becomes a game, then the processor will map the savegame to the normal SRAM location.  I've written all of the code even added a menu for the multi-game but the timing on getting the cart to switch is hit-or-miss.  I've dumped all of my NP carts but it takes multiple tries to get each game done.

I also have to do more testing as I recently got a NP cart with a combination of LoROM and HiROM games and I want to confirm the mapping for the HiROM game.  It appears that the HiROM game is mapped like a LoROM game including the SRAM.  I'll have to do more testing to be sure.

Go Up