[SOLVED] Atari STE - 12-bit using Amiga buffered pickup board

[luciust]

Hi, thanks to @IanSB guidence - I've adopted temprorary Atari ST pickup board intended for Shifter (40-pin) to Atari STE.
The process requirer soldering and will only use 9-bits - not 12-bits, but thanks to the pickup board and bus transcievers - it makes your Atari STE safe from electrical mishaps.
Closeup of the actuall solders:

(Yellow cable is MONO)

HSYNC - right pin & VSYNC on left pin):

Results:
Mono is of course perfect:

Colour - it has some tint - i.e. white has cyan tint, red is a bit wahsed out, but it's far beter then expected:

Let me know if this need more explenation in meantime - I'm waiting for Amiga pickup that will add the 12-bit mode. Also - soldering will always be required, as the clearance for the signals pickup is HARD - my STE has PSU on top of it.

[luciust]

@IanSB
So I've used the Amiga 12-bit pickup board, tested color modes:

• The HSYNC and VSYNC signals have been replaced by Composite Sync from PIN 2 of MC1377 (Color Television RGB to PAL/NTSC Encoder) which is sitting before modulator
• using your 12 bit adapter for RGB2HDMI issue 2
• Mono signal has been attached to PIN of U3 as suggested by you and the chips leg is lifted
• Beta54 FW and STE profile
• RPi Zero and Zero W (testing on two different units)

The outcome:

• in game menu - the picture is exactly the same as with 9-bit signals (0-2 mode) which I tested before - it's darker than original - with colors close to real output - but the colors are completly screwed (dakr green is pink and so on!) - but just during the actual game of Lotus (STE version) - blitter is running then.
• in GEM mode, the while is now white (without cyan tint) and the colors in the GEM menu are OK, but only first 8, where the next 8 are same color as 8 before. On regular monitor - the next 8 colors are lighter - which coresponds with above "darker" picture.

Other:
Minimal color artefacts are visible on some of the parts of picture,

Wondering what might be the issue?

[IanSB]

First check that the sampling menu is set to 12bits per pixel with the STE profile.
Then check that the bits are in the right order
i.e. G3 on STE is connected to G3 on the Amiga pickup board, G2 <> G2, G1<> G1, G0<>G0 etc same for red & blue

[IanSB]

Also need to check your soldering for shorts or unconnected pins

[luciust]

@IanSB
OK, so I went through the connections and so far

• all of the R0-3,B0-3 and G0-3 are connected to RPi GPIO - I've found them from the pickup board up to the RGB2HDMI to RPi header 40 PIN header. They all look like this:
  

The PIN VS/grn1 signal looks like this:

(while the grn1 near the reset switch is the same as other)

The HS:

There's also a fast clock on one of the pins.

Now the color effect - I mentioned - far you can see the "real colors" via the monitor output - the nearer monitor HDMI is the one with the issues:

The lack of "light colors"

The dark image:

And the total mismatch on the colors - if it helps - they are drawn by Billter for sure (contacted author of this port):

I still haven't double-checked if my info on the R-G-B signals is correct and correctly wired - so that is still to do. Lastly - the settings, they are default:

[IanSB]

I still haven't double-checked if my info on the R-G-B signals is correct and correctly wired - so that is still to do

That seems to be the likely cause. If the STE is anything like the ST you need to get the exact schematic for your motherboard because the different revisions of the ST all had different numbering for the resistors so using someone else's info might not work.

The grn1/VS pin will have the mono signal in mono mode but should be inactive in colour modes.

Is there a test pattern program for the STE?
On the Amiga I use the "Amiga Test Kit" and that displays ramps of RGB which makes it immediately obvious which bits are wrong:

[luciust]

Dear @IanSB
Well I've double checked the signals and connections - because there were two sources, ended up in confirming the G0-3 and R-03 bits. Found error - re-soldered the R0-3!

Still I have similar issue!

With 12 bit sampling

• while ingame menu i don't see any activity on R3, B3 or G3 - thus I suspect the picture is always dimmer.
• when ingame play (Lotus STE) the colors are still dim but weird as before - where the blitter is drawing
  With 0-2 sampling
• colors are bright, picture is bright, the blitter parts are OK.
• that is not full palette, but as it seems - at least in menu it is not used 0 meanin bits the 3rd R,G,B bits are not active

3. Thanks to Lopez from local Discord (greetz!) I got the gray tester.
   Look - the 1-3 or 12-bits are the same wrong. But the mode 0-2 is nearly OK, look:

12-bits:

1-3 bits:

0-2 bits:

Could you entertain the idea that code for STE might have some bug? I'm back on Friday to continue double checking the issues - also with simulating missing bits to the imput.

[luciust]

@IanSB The actual board - the MONO hack uses unused socket to meet the patch with mono signal (white cable), the purple is the CSYNC, and R G B are of course RGBs...

[IanSB]

@luciust

Thanks to Lopez from local Discord (greetz!) I got the gray tester.
Look - the 1-3 or 12-bits are the same wrong. But the mode 0-2 is nearly OK, look:

That confirms that you have all the bits rotated by 1
Note that the ST (not STE) is unusual in that you connect the high source bits to R2, G2, B2 and leave R3, G3, B3 used for other things (hence "9 bits (bits 0-2)").
The STE has the more standard connections where the highest bits go to R3,G3,B3.
The STE should look correct when using "9 bits (bits 1-3)" and wrong when using "9 bits (bits 0-2)" but your results are the other way around, hence you have connected STE RGB3 to input RGB2, STE RGB2 to input RGB1 and STE RGB1 to input RGB0.
When you enable full 12 bits per pixel you get alternate dark and light stripes because STE RGB0 is connected to input RGB3 (the brightest part of the signal which alternates on/off with every bar)

So you need to rotate all your RGB connections by 1

[luciust]

@IanSB
That seems to be right, and there's another issue in the C0pperDragon forum.

That also means that the STE Shifter datasheet I got is probably wrong.

Anyway, when I get back I hope to nail it finally, and when sucessfull I'll share the details in FAQ - since I've broken pretty much everything by noe, and found 4 different data on the R,G,B signals :)

Thanks for patience!

[luciust]

For reference: this is the GST Shifter "datasheet" I've been checking my signals with:

[luciust]

Finally - it works!

Based on your suggestions and pixelk0 schema - it works very well:
https://user-images.githubusercontent.com/5732483/124193866-e2011180-dac7-11eb-95ec-914855c7fa58.jpg

[IanSB]

@luciust

Does mono mode work as well?

[luciust]

Yes, takes a little more to kick in - works perefcly (the moire effect is of course only because of camera):

[IanSB]

The genlock is inhibited by default in mono mode because some monitors can't lock above 60Hz and mono mode is 71 Hz. Inhibiting the genlock forces the HDMI output to 60Hz which means you should always get an image on the screen.
If your monitor can lock to 71Hz you can try changing the refresh to "Force 50Hz-Any" which should allow the HDMI output to always track the frequency of the input.
Use menu+reset to recover if this doesn't work.

[luciust]

Good idea, I got two more make-SHIFT HDMI monitors - fun thing to try!