WARNING — THIS TUTORIAL IS NOT COMPLETE. I am still learning how to use WordPress. Part of this tutorial is for me to learn the process and put a system in place for future posts and videos. At this time my plan is to write the blog post and then create a video to do with it.
This tutorial will show how to transfer Camera images from an OPEN-MV H7 to a Nextion NX4827P043_011 display. I will be using two ESP32s to accomplish this tutorial. The display will configure the WIFI connection and display the image. The ESP receiving the image from the camera will have a static IP address. The Camera will have a dynamic address but be programmed with the static address in the ESP32. This configuration may not be ideal, but this tutorial is about transferring the image and only a little bit about the configuration of the network.
- I will start in the Nextion Display
- Then move to the OPEN-MV Camera
- Finish with the ESP32
Below are the parts used in this tutorial. If you don’t need the parts but want to support me, you can enter any product’s ASIN number into the input field and click the link. You will open an Amazon page with the link, and the credit will be given to Cheap Controls.
Nextion Code
The Nextion display will have three pages. The first page will be a simple page to select the configuration or the image display. the second page will have buttons to get the available SSIDs and populate the empty drop-down box. The second page will also have a button to connect to the selected SSID using the password entered into t0. The third page will display the image. The RESET and GET PIC buttons will send commands, but I am not utilizing them in this tutorial. If you Notice the NET and CAM areas on each page. I have the NET field configured to show if the WIFI is connected. I plan to add the CAM indicator to show if the camera is connected. These would be indicator lights only.
I am going to go over the highlights of the Nextion code below. You can download all the files at the end of this blog post.
PAGE 00
CONFIG button - on release
prints "C:CPAG01?",0
page 1
DISPLAY button - on release
prints "C:CPAG02?",0
page 2
PAGE 01
GET SSIDs button - on release
prints "C:CGET10?",0
CONNECT button - on release
prints "C:CCON",0
prints cb0.txt,0
prints ":",0
prints t0.txt,0
prints "?",0
HOME button - on release
prints "C:CPAG00?",0
page 0
PAGE 02
RESET button - on release
prints "C:CRST01?",0
GET PIC button = on release
prints "C:CPIC01?",0
HOME button - on release
prints "C:CPAG00?",0
page 0 OPEN-MV Camera Code
I have to be honest. I am still learning Python, so some things in this portion could be done better. Please refer to the notes in the code. This is the link to the camera CLICK HERE. I have no connection to the OPEN-MV camera line.
# MJPEG Streaming
#
# This example shows off how to do MJPEG streaming to a FIREFOX webrowser
# Chrome, Firefox and MJpegViewer App on Android have been tested.
# Connect to the IP address/port printed out from ifconfig to view the stream.
import sensor, image, time, network, usocket, sys , ubinascii
#EDIT THE WIFI
SSID="Engineering" # Network SSID
KEY="testtest" # Network key
HOST ='' # Use first available interface
PORT = 80 # Arbitrary non-privileged port
# Reset sensor
sensor.reset()
# Set sensor settings
sensor.set_contrast(1)
sensor.set_brightness(1)
sensor.set_saturation(1)
sensor.set_gainceiling(16)
sensor.set_framesize(sensor.QVGA) # There alternates to QVGA
sensor.set_pixformat(sensor.RGB565)
sensor.skip_frames(time = 2000) # Let new settings take affect.
# Init wlan module and connect to network
print("Trying to connect... (may take a while)...")
wlan = network.WINC()
wlan.connect(SSID, key=KEY, security=wlan.WPA_PSK)
# We should have a valid IP now via DHCP
print(wlan.ifconfig())
# Create server socket
s = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM)
# The IP address I used below is arbitrary but must match the ESP
addr = usocket.getaddrinfo('192.168.4.150', 80)[0][-1]
s.setblocking(True)
s.connect(addr)
print ('Connected to ' + addr[0] + ':' + str(addr[1]))
# FPS clock
clock = time.clock()
pageNum = 0
# Start streaming images
# NOTE: Disable IDE preview to increase streaming FPS.
frame = sensor.snapshot() # Takes the photo
cframe = frame.compressed(quality=10) #
header = "Content-Length:"+str(cframe.size())
#s.send(header)
while (True):
clock.tick() # Track elapsed milliseconds between snapshots()
# wait for ESP32 to signal for new image
sig = str(s.readline())
# used to get the complete command string minus transmission characters
# I remove the first 2 and last 3 characters
print(sig[2:(len(sig)-3)])
# Extract the command
ESPcommand = sig[5:8]
# Extract the value
ESPvalue = (sig[8:(len(sig)-3)])
# if there is a page change change the pageNum variable
if ESPcommand=='PAG':
print("PAGE CHANGE")
print(ESPvalue)
pageNum = 0
if ESPvalue=='02':
print("SEND IMAGE")
pageNum = 2
# if on page 2 send updated photos repeatedly
if pageNum == 2:
frame = sensor.snapshot() # Takes the photo
cframe = frame.compressed(quality=10) #
header = "Content-Length:"+str(cframe.size())
s.send(header)
print(s.send(ubinascii.hexlify(cframe)+"\r"))
# if not on page 2 send the text "PING"
# I found this helps keep the connection between devices working
if pageNum != 2:
s.send("PING\r")ESP32 Code
There are four separate files used in this project. The main file contains the settings and variables. I have a standard configuration for the Nextion Displays. I have a endChar string that is used every time I send a command to the Display. I have a dfd string that contains the data received from the display.
// NOTE : Below is a String I use to add to the data sent to the Nextion Display
// And to verify the end of the string for incoming data.
// Replace 3 x Serial2.write(0xff);
const String endChar = String(char(0xff))+String(char(0xff))+String(char(0xff));
String dfd = ""; // data from displayThe next step is to create the variables for a general asynchronous delay. This comes in handy if you want something to happen at regular intervals. I start by flashing the onboard LED. this helps with the initial configuration. I can change the “delayLength” variable and make sure the upload process is working.
// NOTE : General Async Delay
unsigned long asyncDelay = 0;// NOTE : 4,294,967,295
int delayLength = 10000;I configure the network parameters. I leave the SSID and PASSWORD configuration in the Nextion. This is not the best way because I use a static IP address. A proper address to connect to the DHCP address that will be configured in the OPEN-MV camera. Since this tutorial is about the image transfer I will try to address a better way to configure the IP settings in another tutorial.
// Network Variables
String SSIDname = "NoName";//CheapControls -- Engineering
String SSIDpassword = "NoPass";//controls -- testtest
IPAddress staticIP(192, 168, 4, 150);
IPAddress gateway(192, 168, 4, 1);
IPAddress subnet(255, 255, 255, 0);
IPAddress primaryDNS(8, 8, 8, 8);
IPAddress secondaryDNS(8, 8, 4, 4);
int connStatus = 0;
WiFiServer wifiServer(80);
word currentMillis = 0;
String currentPage = "00";
String newPage = "00";
Next I have the variables used in the file transfer between the ESP32 and the display
//NOTE : twfile and stuff
String payload;
int picLength;
int totalBytes = 0;
int packetNum = 0;
int packetLength = 1000;
int twfileDog = 0;
int sendImage = 0;The setup portion is very short. There is a serial port for informational messages sent to the Serial Monitor. A serial connection to the display, note the 921,600 baud rate. A higher baud rate is required due to the size of the picture files. I also configure the onboard LED for troubleshooting.
void setup() {
Serial.begin(9600);
Serial2.begin(921600);
pinMode(2,OUTPUT);
}// NOTE : These functions need to be checked if the camera is connected or not
// They are called inside the while loop and outside the while loop
void generalFunctions(){
// NOTE : COLLECT CHARACTERS FROM NEXTION DISPLAY
if(Serial2.available()){
inputFunction();
}
// NOTE : ASYNC DELAY
if(millis() > asyncDelay){
asyncDelay+=delayLength;
delayFunction();
}
//NOTE : SOMETHING SENT FROM NEXTION I.E. GET REQEUEST
if(dfd.endsWith(endChar)){
Serial.println(dfd);
Serial.println("error");
dfd = "";
}
}#include <Arduino.h>
#include <WiFi.h>
#include <String.h>
// NOTE : Below is a String I use to add to the data sent to the Nextion Display
// And to verify the end of the string for incoming data.
// Replace 3 x Serial2.write(0xff);
const String endChar = String(char(0xff))+String(char(0xff))+String(char(0xff));
String dfd = ""; // data from display
// NOTE : General Async Delay
unsigned long asyncDelay = 0;// NOTE : 4,294,967,295
int delayLength = 10000;
// Network Variables
String SSIDname = "NoName";//CheapControls -- Engineering
String SSIDpassword = "NoPass";//controls -- testtest
IPAddress staticIP(192, 168, 4, 150);
IPAddress gateway(192, 168, 4, 1);
IPAddress subnet(255, 255, 255, 0);
IPAddress primaryDNS(8, 8, 8, 8);
IPAddress secondaryDNS(8, 8, 4, 4);
int connStatus = 0;
WiFiServer wifiServer(80);
word currentMillis = 0;
String currentPage = "00";
String newPage = "00";
//NOTE : twfile and stuff
String payload;
int picLength;
int totalBytes = 0;
int packetNum = 0;
int packetLength = 1000;
int twfileDog = 0;
int sendImage = 0;
void setup() {
Serial.begin(9600);
Serial2.begin(921600);
pinMode(2,OUTPUT);
}
void loop() {
if(WiFi.status()!=3){
generalFunctions();
connStatus = 0;
}else{
if(connStatus == 0){
Serial.println("attempting connection...");
Serial.println(SSIDname + " : " + SSIDpassword);
Serial.println(WiFi.localIP());
Serial.println("STATUS : " + String(WiFi.status()));
connStatus = 1;
}
WiFiClient client = wifiServer.available();
if (client) {
Serial.println("Client connected...");
while (client.connected()) {
// NOTE : ASYNC DELAY
if(millis() > asyncDelay){
asyncDelay+=delayLength;
delayFunction();
}
// NOTE : Keeps contact to camera when not sending image
// Camera will timeout if 10 seconds goes by with out some sort of signal
if(millis()>currentMillis && currentPage!="02"){
client.write("PING\n");// NOTE : "PING" could be any text
currentMillis+=5000;
}
// NOTE : Sends another image if still on page 2 and current image has been sent
if(currentPage == "02" && sendImage == 1){
client.write("C:CSND\n");
sendImage=0;
}
// NOTE : COLLECT CHARACTERS FROM NEXTION DISPLAY
if(Serial2.available()){
inputFunction();
}
// NOTE : Send page Nextion page change to Camera
// this might not be neccessary in finished product
if(currentPage != newPage){
client.write("C:CPAG");
client.write(newPage.c_str());
client.write("\n");
currentPage = newPage;
}
// NOTE : collect data from camera
while (client.available() > 0) {
char c = client.read();
if (c != '\r') {
payload += c;
} else {
Serial.println("-------------------START PAYLOAD----------------");
Serial.println("payload size = " + String(payload.length()));
if(payload.length() < 50){
Serial.println(payload);
payload="";
Serial.println("---------------END SHORT PAYLOAD----------------");
}
else{
Serial.println("Very large Payload");
// NOTE : next two lines helkp to parse out the image from the payload
int startindex = payload.indexOf("ffd8");
int endindex = payload.indexOf("ffd9");
// NOTE : Extract the image from the payload
payload = payload.substring(startindex, endindex + 4);
picLength = payload.length()/2; // NOTE : image lenght is half the size of the payload
Serial.println("Payload Length = " + String(payload.length()) + " -- image size = " + String(picLength));
//NOTE : Send the initial twFile command to this Nextion display.
// this will start the process and then switch to input tab based on replies from the display
Serial2.print("twfile \"sd0/img.jpg\"," + String(picLength) + endChar);
Serial.println("twfile \"sd0/img.jpg\"," + String(picLength));
//NOTE : We wait for a response from the display before sending the data
Serial.println("---------------END LONG PAYLOAD----------------");
}
}
}
}// NOTE : end of while connected
Serial.println("Client disconnected");
}// NOTE : end of if client
}
}
// NOTE : These functions need to be checked if the camera is connected or not
// They are calle inside the while loop and outside the while loop
void generalFunctions(){
// NOTE : COLLECT CHARACTERS FROM NEXTION DISPLAY
if(Serial2.available()){
inputFunction();
}
// NOTE : ASYNC DELAY
if(millis() > asyncDelay){
asyncDelay+=delayLength;
delayFunction();
}
//NOTE : SOMETHING SENT FROM NEXTION I.E. GET REQEUEST
if(dfd.endsWith(endChar)){
Serial.println(dfd);
Serial.println("error");
dfd = "";
}
}void delayFunction(){
digitalWrite(2,!(digitalRead(2)));
// NOTE : STUFF HERE
//Serial.println(SSIDname + " : " + SSIDpassword);
//Serial.println(WiFi.localIP());
//Serial.println("STATUS : " + String(WiFi.status()));
// 0: WL_IDLE --temp if WiFibegin is called
// 1: WL_NO_SSID_AVAIL No SSID avail
// 2: WL_SCAN_COMPLETE Network SSID scan complete
// 3: WL_CONNECTED connected to SSID
// 4: WL_CONNECT_FAILED failed to connect to SSID
// 5: WL_CONNECTION_LOST connection is lost
// 6: WL_DISCONNECTED disconnected from network
if(WiFi.status()==3){
Serial2.print("net.bco=1024" + endChar);
}else{
Serial2.print("net.bco=63488" + endChar);
}
// NOTE : RESET twfile state if timeout
if(packetNum > 0){
twfileDog++;
Serial.println("twFile Watch dog is now " + String(twfileDog));
}
if(twfileDog > 5){
Serial.println("Need to Reset Display after a timeout");
resetHeader();
twfileDog=0;
packetNum=0;
delay(100);
//Serial2.print("b3.bco=63488" + endChar);// NOTE: set to red
}
}void resetHeader(void){
sendHeader(0xFFFF,0);
}
void sendHeader(int packetNum, int packetLen){
char header[] = {0x3A, 0xA1, 0xBB, 0x44, 0x7F, 0xFF, 0xFE, 0x00, 0x00, 0x00, 0x00, 0x00};
header[8] = (byte)(packetNum) & 255;
header[9] = (byte)((packetNum) >> 8) & 255;
header[10] = (byte)(packetLen) & 255;
header[11] = (byte)((packetLen) >> 8) & 255;
for (int h = 0; h < 12; h++){
//Serial.print((byte)header[h], HEX);
//Serial.print("-");
Serial2.write(header[h]);
}
//Serial.println();
}
void sendPacket(void){
// NOTE : Image data is received as nibbles, we need a temp byte to combine nibbles
// into bytes
byte tempByte = 0;
packetLength = 1000;
sendHeader(packetNum,packetLength/2);
for(int x=0;x<packetLength;x++){
int picPointer = (packetNum*packetLength)+x;
byte tempNibble = payload[picPointer] < 'A' ? payload[picPointer] & 0xF :(9 + (payload[picPointer] & 0xF));
// NOTE : If picPointer is an even number we need to shift the nibble
// If picPointer is an odd number then we need to combine it with the previous
// shifted value and write the value
if(picPointer%2 == 0){
tempByte = tempNibble*16;
}else{
Serial2.write(tempByte | tempNibble);
}
}
packetNum++;// NOTE : initially set on main tab and reset on input of 0XFD
}void inputFunction(){
dfd += char(Serial2.read());
if((dfd.length()==1) && (byte)dfd[0] == 0x05){
//Serial.println("0x05(5) Send packet #" + String(packetNum) + " packet Length = " + String(packetLength));
sendPacket();
dfd="";
}
if((dfd.length()==4) && (dfd.endsWith(endChar))){
// NOTE : 0x06 is code for invalid file name
if((byte)dfd[0] == 0x06){
Serial.println("0x06 invalid file name");
}
// NOTE : 0x1A(26) is code for invalid variable
if((byte)dfd[0] == 0x1A){
Serial.println("0x1A(26) invalid variable -- General error");
}
// NOTE : FE - display is ready for data 0xFE is decimal 254
if(((byte)dfd[0] == 0xFE)){
//Serial.println("0xFE(254) - file created - Time to Load Data");
sendPacket();
// NOTE : Should get a 0xFD response if successfull
}
// NOTE : FD -> Nextion has received the file correctly is decimal 253
if((byte)dfd[0] == 0xFD){
//Serial.println("0xFD(253) - file received correctly");
delay(50);
twfileDog=0;
packetNum=0;
payload="";
sendImage=1;
//Serial2.print("b3.bco=1024" + endChar);// NOTE: set to green
}
dfd="";
}
if(dfd.length()>4 && dfd.substring(0,3)!="C:C") dfd="";
else{
if(dfd.substring((dfd.length()-1),dfd.length()) == "?"){
String command = dfd.substring(3,6);
String value = dfd.substring(6,dfd.length()-1);
Serial.println(command + " : " + value);
if(command == "GET"){
Serial2.print("xstr 0,110,320,40,0,BLACK,0,1,1,3,\"SCANNING...\"" + endChar);
int n = WiFi.scanNetworks();
Serial.println("Number of Networks found = " + String(n));
String SSIDs = "";
for (int i = 0; i < n; ++i) {
if(i > 0)SSIDs += "\r\n";
SSIDs += WiFi.SSID(i);
Serial.println(WiFi.SSID(i));
}
Serial2.print("config.cb0.txt=\"" + String(n) + " Networks\"" + endChar);
Serial2.print("config.cb0.path=\"" + SSIDs + "\"" + endChar);
Serial2.print("xstr 0,110,320,40,0,WHITE,0,1,1,3,\"SCANNING...\"" + endChar);
}
if(command == "CON"){
int colonLoc=value.indexOf(":");
SSIDname = value.substring(0,colonLoc);
SSIDpassword = value.substring(colonLoc+1);
WiFi.config(staticIP, gateway, subnet, primaryDNS, secondaryDNS);
WiFi.begin(SSIDname.c_str(),SSIDpassword.c_str());
Serial.print("Connected to WiFi at address:");
Serial.println(WiFi.localIP());
wifiServer.begin();
}
if(command == "PAG") newPage = value;
dfd="";
}
}
}