HOWTO? debug using JTAG ICE when LCD shield with touchscreen is attached

I have the "Arduino Mega2560 R3 compatible" SainSmart C46 Kit with Mega2560 + 3.2" TFT LCD Display + TFT LCD Adjustable Shield stack (see http://www.amazon.com/gp/product/B008FWSH3C/ref=oh_details_o00_s00_i00?ie=UTF8&psc=1 for details) that I've been programming for a customer application.

It seems I ran out of RAM because my finite state machine (FSM) objects were being stored in RAM rather than PROGMEM. I rewrote the code to put the FSM objects in PROGMEM, but now my code doesn't work. It's complex, and I need to stop doing "by guess and by golly" debugging. I tried to use Visual Micro, but it doesn't stop on breakpoints, and I haven't been able to get any debugging done with it. I bought an Atmel AVR JTAG ICE3 (Smart | Connected | Secure | Microchip Technology), but it REQUIRES pins 4-7 be available for the JTAG interface, and I think those are used by the touchscreen in this stack (that's what the code seems to indicate). I don't know for certain, though, because I can't seem to find schematics for the LCD or adjustable adapter shield anywhere.

Does anyone have a clue of where to get schematics for the SainSmart LCD and adapter shields? I've been trying to post questions and messages in their forums, but all of my posts are still marked as "pending admin approval" so no one can see them other than me - which means I'm not getting any answers there.

Has anybody gotten the JTAG ICE to work with a touchscreen LCD connected to the Arduino Mega2560 R3? Is it possible to make this work, or do I have to hack up the boards? If I have to do hardware modifications, where can I find information about how to do it correctly?

I've got a lot of experience doing hardware and software development, but the horrifying lack of documentation here is making it nearly impossible to proceed!

see attached file

TFT01_Mega_shield.pdf (45.8 KB)

Thanks for sending this - it's more than I've found so far. Unfortunately, I'm pretty sure it doesn't quite match the board I've got which has 4 zero ohm resistors, this schematic only shows one. Based on comments I've seen elsewhere, I'm guessing the SD_SCK and SD_CS lines have two of the different ones: This must be a V1.0 (version 2) board. SainSmart | Desktop CNC, 3D Printing & DIY Tools | Power to the Makers – SainSmart.com

If anybody comes across this page automation.binarysage.net is Expired or Suspended. while searching for Arduino Mega R3 JTAG debugging information please note that the Arduino pinout information is WRONG:

ATmega2560 JTAG Pinouts

Because the 2560 is part of the larger range of devices from Atmel, this unit has a JTAG interface. On smaller devices, debugging is handled with an interface called ‘DW’ or ‘Debug Wire’. This is a pin saving diagnostic interface that uses the Reset pin on the device as a debug control signal. However, on the larger devices, a full JTAG interface is provided to diagnostic purposes.

There is a minor point to highlight and that is the JTAG interface uses 4, yes 4, of the ADC (Analogue to Digital Convert) pins. Once JTAG is enabled in the 2560, these 4 ADC pins can not be used for reading analogue signals and should not be connected to anything other than the JTAG port of the AVR Dragon.

There are only 6 connections needed and these are as follows:
Pin Signal I/O Description
5v +5v Output 5v Supply from Arduino
GND GND - Ground reference
4 TCK Input Test Clock, clock signal from AVR Dragon to target JTAG port
5 TMS Input Test Mode Select, mode select signal from AVR Dragon to target JTAG port
6 TDO Output Test Data Output, data signal from target JTAG port to AVR Dragon
7 TDI Input Test Data Input, data signal from AVR Dragon to target JTAG port

Note: These pin numbers refer to the shield pin numbers of the Arduino Mega2560 and not to the pins on the processor.

Referring to the ATmega2560 spec sheet http://www.atmel.com/images/doc2549.pdf? and the Arduino Mega 2560 R3 schematic http://arduino.cc/en/uploads/Main/arduino-mega2560-schematic.pdf?, we see the correct JTAG pins are:

JTAG
Signal
Atmel Pin Arduino
Signal
Connector
Position
TDI 90 ADC7 8
TDO 91 ADC6 7
TMS 92 ADC5 6
TCK 93 ADC4 5

Be careful about following misinformation from the Web or you’ll end up doing difficult repairs with 28-gauge wire like I did:

http://theoriginalprototype.com/projects/Arduino/20140316-wfredk-hardware-repair.jpg

(Sorry, I couldn’t figure out how to put the image directly into this post…)

The blog is correct. I don't know where you got that ADC4 is mapped to Arduino Pin 5 and ADC5 is mapped to Pin 6, etc. Refer to http://arduino.cc/en/Hacking/PinMapping2560
In fact, I was able to debug both of my Arduino Mega boards (an old Arduino Mega 1280 and a new Arduino Mega 2560 R3) connecting ADC Pins 4-8 to an AVR Dragon, based on the blog's instructions.

If you look at a Mega board, you see that the pins that are numbered 4-7 are DIGITAL pins, on the PWML interface, as shown on the schematic.

You are correct in that ANALOG pins 4-7 are the JTAG pins - which are pins 5-8 on the CONNECTOR - again, refer to the schematic.

I'm glad you can debug your boards using your AVR Dragon as per the blog post - my question was about connecting an AVR JTAGICE3 - which is a completely different machine.