Waveshare ESP32-S3-ETH support

[stanelie]

How do I go about having CircuitPython work on this device?

Wanting to do some ethernet to ESPnow, I got myself a Waveshare ESP32-S3-ETH, a ESP32-S3 device with a W5500 WIZnet ethernet port. I only now realize that this board is not supported by CircuitPython. I've tried to install a few of the ESP32-S3 builds for other devices on it, without luck.

Here is the wiki link for it : https://www.waveshare.com/wiki/ESP32-S3-ETH
Schematic : https://files.waveshare.com/wiki/ESP32-S3-ETH/ESP32-S3-ETH-Schematic.pdf

According to the schematic, the W5500 ethernet device is a SPI device connected to the following ports :

GP9 : RST
GP10 : INT
GP11 : MOSI
GP12 : MISO
GP13 : SCLK
GP14: SCSn

[dhalbert]

If you're interested in trying this, see
https://learn.adafruit.com/how-to-add-a-new-board-to-circuitpython
and for general background on doing builds:
https://learn.adafruit.com/building-circuitpython

The easiest thing would be to copy an existing W5500-capable board and modify as needed.

[stanelie]

There are a few boards with a W5500 interface, but they are RP2040 based. Would that work?
Do you think I have enough info with the wiki page from waveshare to attempt this? I'm just a simple user, not a dev...

[bablokb]

You will actually have to merge a suitable ESP32-S3 board definition with a W5500 board-definition. You need the basic setup including source-code position below ports/espressif/boards from the first and the SPI-definitions (adapted for your pins) from the second.

[stanelie]

I will require some hand holding...
Trying to setup the build environment on Ubuntu 22.04, I run into this issue : on the page https://learn.adafruit.com/building-circuitpython/espressif-build , it says to

cd circuitpython/ports/espressif
esp-idf/install.sh

However, the install.sh script is missing...
Thoughts?

[bablokb]

I always recommend to use the instructions from .devcontainer/Readme.md. This creates a complete dev-environment in the cloud automatically and you don't have to mess with installing everything yourself.

[stanelie]

Ok, I did that.
I am now encountering some errors trying to compile for a device I own (so I can test the setup is working correctly). How do I go from here? Should I continue in this thread or elsewhere?

-- Found assembler: xtensa-esp32s3-elf-gcc
CMake Error at esp-idf/tools/cmake/project.cmake:564 (__project):
  The CMAKE_C_COMPILER:

    xtensa-esp32s3-elf-gcc

  is not a full path and was not found in the PATH.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.
Call Stack (most recent call first):
  CMakeLists.txt:12 (project)


CMake Error at esp-idf/tools/cmake/project.cmake:564 (__project):
  The CMAKE_CXX_COMPILER:

    xtensa-esp32s3-elf-g++

  is not a full path and was not found in the PATH.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.
Call Stack (most recent call first):
  CMakeLists.txt:12 (project)


CMake Error at esp-idf/tools/cmake/project.cmake:564 (__project):
  The CMAKE_ASM_COMPILER:

    xtensa-esp32s3-elf-gcc

  is not a full path and was not found in the PATH.

  Tell CMake where to find the compiler by setting either the environment
  variable "ASM" or the CMake cache entry CMAKE_ASM_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.
Call Stack (most recent call first):
  CMakeLists.txt:12 (project)


-- Warning: Did not find file Compiler/-ASM
-- Configuring incomplete, errors occurred!
make: *** [Makefile:555: build-waveshare_esp32_s3_zero/esp-idf/config/sdkconfig.h] Error 1
@stanelie ➜ /workspaces/circuitpython/ports/espressif (main) $ 

[stanelie]

found the missing compiler and added it to the path with
export PATH=/home/codespace/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin:$PATH
Now, I get a new error :

CMake Error at esp-idf/tools/cmake/build.cmake:552 (message):
  /home/codespace/.python/current/bin/python: Error while finding module
  specification for 'idf_component_manager.prepare_components'
  (ModuleNotFoundError: No module named 'idf_component_manager')

[jepler]

The commands are added o the path when you "source" the "export.sh" script" (from the espressif specific page of the guide linked above):

After this, in each fresh terminal window in which you are doing builds, you need to use esp-idf/export.sh in order to set up the correct PATH and other environment variables.
Note that export.sh sets up a Python venv (virtual environment). If you are already using a venv, maybe because you're using a recent Debian or Ubuntu version of Linux, you'll need to deactivate that venv before running export.sh. Otherwise you will get an error that you can't create venv inside another venv.

# Do this in each new terminal.
cd circuitpython/ports/espressif
source esp-idf/export.sh

note that you have to also install circuitpython's requirements into the virtual environment that export.sh uses, a step that appears to be missed in that document. (I submitted a feedback note about it)

[bablokb]

Ok, I did that.

I am not sure. If you follow the instructions verbatim, you won't have any problems with missing compilers, environments and so on. It is just a number of clicks and then you are ready to build your board.

[stanelie]

Thanks for this info!
I managed to build a firmware for my ESP32-C3 device (as a test) and load the .bin on an actual Maker Go ESP32C3 Supermini. I don't see the virtual disk drive, but it boots and shows up in Thonny. Great!

Now, which device should I clone as a starting point for this Waveshare ESP32-S3-ETH device? I'm guessing the kind of memory chip for the rom is important, as well as the address for it?
The wiki page for this device on the waveshare website says it has a 16 MB flash size, and the chip on my desk is a Winbond 250128JVSO (SPI I believe).

[stanelie]

P.S.
for this line, I'm unsure how to set it to english :
"make -j $(nproc) BOARD=whatever TRANSLATION=xx_XX"

[anecdata]

@stanelie -C3 boards do not have native USB, so do not have a CIRCUITPY drive.

Try TRANSLATION=en_US

[anecdata]

You do want to match chip/module, and flash and RAM. UnexpectedMaker ProS3 (and FeatherS3) boards have 16MB flash and 8MB RAM, probably others too. Pins and special features will differ. You'll also need VID/PIDs from Waveshare (or https://github.com/espressif/usb-pids).

The W5500 part doesn't involve much, support over SPI is provided by a CircuitPython library. But you could make that SPI the default board.SPI(), and add pin aliases for the W5500 interface pins.

It's possible to freeze in a CircuitPython library (e.g., WIZnet), but realize that it will take precedence over a same name library in the /lib folder (but not over the same name library at /). I don't know whether this is generally recommended or not.

[stanelie]

I've just tried flashing the UnexpectedMaker ProS3 firmware to the ESP32-S3-ETH, and it doesn't go any further then creating the virtual boot drive (PROS3BOOT, in this case). I am unable to load the .uf2 file. What should I try next?

[stanelie]

Does this help? It's from the setup instructions for the Arduino IDE :

[anecdata]

the waveshare may not have the UF2 bootloader, you could use esptool or the web tool to load the .bin firmware (or install a UF2 bootloader) ... edit: I guess it does since you get the boot volume

what happens when you drag the UF2 to PROS3BOOT?

[stanelie]

It looks as though it is copying the file to the PROS3BOOT drive, and when done, it dieconnects the drive, but when it comes back, it is still the PROS3BOOT drive, and I cannot connect REPL

[stanelie]

The UnexpectedMaker PRO S3 does indeed have the same memory chip, as I found on this picture. Exact same marckings on the Winbond chip :
https://www.tinytronics.nl/image/cache/catalog/products_2024/unexpected-maker-pros3-esp32-s3-development-board-usb-c-1-1500x1500.jpg

[anecdata]

best guess is there's some incompatible difference in the ProS3 UF2. I don't know if it matters, but ProS3 has 80MHz QIO for flash and RAM, Waveshare has octal PSRAM

there's another board that may match better: yd_esp32_s3_n16r8 QIO flash, OIO psram, although it's using a WROOM-1 module)

[stanelie]

In case this is useful, here is what the stub says when connecting to the board from the esptool.py flash page:

Stub is now running...
Detecting Flash Size
FlashId: 0x1840EF
Flash Manufacturer: ef
Flash Device: 4018
Auto-detected Flash size: 16MB

I will try this other dev board firmware

[stanelie]

Oh!!
The yd_esp32_s3_n16r8 works, it boots fully! I have a CIRCUITPY drive and REPL is connected! Woohoo!
Thanks @anecdata !!

[stanelie]

So, I'm not having much success. After much faffing about, I decided to restart from the beginning.

I built the current yd_esp32_s3_n16r8 in the online build environment with the line
time make -j 2 BOARD=yd_esp32_s3_n16r8 TRANSLATION=en_US . After a while, it spits out a firmware.bin in the build directory, and I use https://adafruit.github.io/Adafruit_WebSerial_ESPTool/ to flash it to my ESP32-S3-ETH board. It does not work.

If I flash the .bin file from the circuitpython website, it works.
Shouldn't these 2 .bin files be identical?

So... I'm a bit at a loss here... Thoughts?

[stanelie]

Still can't build the firmware myself, but I can reiterate that the yd_esp32_s3_n16r8 firmware works very well, even if there are a lot of missing IO pins and it is mislabeled. I was able to make the Wiznet W5500 run perfectly, responding to ping and pulling web pages from the net.

Initialization of the onboard W5500 :

cs = digitalio.DigitalInOut(board.GPIO14)
spi_bus = busio.SPI(board.GPIO13, MOSI=board.GPIO11, MISO=board.GPIO12)

I also had Espressif generate a PID USB code for this board :
0x82A7 | Waveshare ESP32-S3-ETH - Circuitpython

[anecdata]

In what way does the built firmware not work... errors during build, no serial, no CIRCUITPY, boot looping, etc.?

Perhaps if you're up for submitting a draft PR, folks could help spot the issue.

[stanelie]

The build for the yd device completes without errors. I then flash the firmware on the actual board, and then, nothing. No serial, no drive, it makes my keyboard and mouse lag every few seconds, I have to disconnect the board from my computer to keep working. Where should I send my draft files?

[anecdata]

It's possibly boot-looping, but you may have to connect to separate debug part pins (if exposed) to see it? (that's the way it used to be, not sure if it still is)

If you're using the devcontainer process, maybe bablokb or someone else can help with how to turn that into a PR.

[stanelie]

Yes, I've been using the codespace as suggested by @bablokb. I thought that building a firmware in the codespace build environment was the same as the automated system used by the Circuitpython community to generate the builds for every supported board. I looked at the firmware I got and compared it to the one from the Circuitpython download page with a hex editor, they are not the same at all!

[stanelie]

Here are my draft files.
waveshare_esp32_s3_eth.zip

[bablokb]

I looked at the firmware I got and compared it to the one from the Circuitpython download page with a hex editor, they are not the same at all!

They will never be. The files on the download page are created from a specific commit, you are probably building from the current state of CircuitPython. Depending on what went in since the last official download version, the bleeding edge version might or might not work. But it will definitely be different.

Some things to check: size of the bin-file in the devcontainer and after downloading. If sizes match, you could also run md5sum on both versions, just to make sure there was no problem during download. You could also post the summary lines after the build finishes (with the memory sizes of the file).

[bablokb]

I thought that building a firmware in the codespace build environment was the same as the automated system used by the Circuitpython community to generate the builds for every supported board.

This might or might not be true. The devcontainer-builds follow the instructions from the document linked in the post above by Dan. It is more or less only a scripting solution for the steps he describes.

The automated builds for publishing use their own scripts. Since I never read anything about changes needed for productive builds, I am assuming that the scripts from the devcontainer also create production-level builds. But I might be wrong here.