Getting Out of the Echo Chamber – Transferring Data Over Ethernet Using Zynq

We recently revamped a tutorial on the Digilent Reference site which details how to set up an LWIP echo server for Zynq-7000 boards: Getting Started with Zynq Servers. This post is something of an appendix to that tutorial, showcasing some simple changes that one can make to the server to programmatically send data back to a client. The tutorial includes basic manipulation of echoed data – this post takes it a step farther, generating data in the server software application, which could be trivially extended to arbitrary data, for example, data captured from a sensor or other device hooked up to the FPGA. This post assumes that you have already run through the tutorial, however, Zynq presets make it so that you can get started pretty much right away using any hardware platform (XSA) you already have sitting around – do check the tutorial for some bug fixes required by some boards. 

Sending a block of data to the host 

First, we’re going to make some changes to the LWIP echo server so that it can send back custom data – not just echoing what was sent to it. In this case, the server will accept a connection from a client, receive a packet from it, which consists only of an integer number of bytes to send back, and then return that much data. Importantly, a single TCP packet can only transmit so many bytes. We’re going to implement both sides of the connection such that “one transfer” can be spread across multiple tcp_writes and recvs. 

All of the important code in the LWIP echo server example can be found in echo.c. 

The LWIP TCP API uses a simple callback model to take action when various Ethernet events occur. A tcp_accept event occurs when a tcp connection is first established between client and server. In the base LWIP server example, this callback just registers another callback, tcp_recv, so that whenever any data is received by the server from the client, it can take further action. We’re going to modify it to also register a tcp_sent callback, which will be used to take further action when a packet is successfully sent from the server to the client, allowing us to send more data than can fit in any single packet. 

Note: The API also includes a tcp_err callback which can be used to take action when a fatal error occurs on the connection, however, that won’t be used here.

err_t accept_callback(void *arg, struct tcp_pcb *newpcb, err_t err) 
{ 
    static int connection = 1; 

    /* set the receive callback for this connection */ 
    tcp_recv(newpcb, recv_callback); 

    /* set the sent callback for this connection */ 
    tcp_sent(newpcb, sent_callback); // <- add this 

    /* just use an integer number indicating the connection id as the 
       callback argument */ 
    tcp_arg(newpcb, (void*)(UINTPTR)connection); 

    /* increment for subsequent accepted connections */ 
    connection++; 

    return ERR_OK; 
} 

Next, we’re going to add some global variables to pass data between the callback functions. Test_buf holds the data to be sent to the client. Bytes_to_send represents the number of bytes remaining in the buffer which have yet to be sent. Write_head will track the first address within the buffer where there is a byte which still needs to be sent. These should be added near the top of the file, so that they can be accessed from within the callback functions. 

// global variables used to pass data between callbacks, tcp_arg
// could potentially be used instead 
u16_t bytes_to_send;
u8_t test_buf[65536];
u8_t *write_head;

We’ll also add a helper function that will be used to add data to the LWIP library’s send buffer and keep track of our own buffer by updating the global variables. Note that tcp_sndbuf returns the available space left in the transmit buffer.

err_t push_data(struct tcp_pcb *tpcb)
{
    // add up to remaining bytes_to_send to the transmit buffer, or
    // fill the remaining space in the transmit buffer
    u16_t packet_size = bytes_to_send;
    u16_t max_bytes = tcp_sndbuf(tpcb);
    err_t status;

    // if there's nothing left to send, exit early
    if (bytes_to_send == 0) {
        return ERR_OK;
    }

    // if adding all bytes to the buffer would make it overflow,
    // only fill the available space
    if (packet_size > max_bytes) {
        packet_size = max_bytes;
    }

    // write to the LWIP library's buffer
    status = tcp_write(tpcb, (void*)write_head, packet_size, 1);

    xil_printf("push_data: Asked to add %d bytes to the send buffer, adding %d bytes\r\n",
               bytes_to_send, packet_size);

    // keep track of how many bytes have been pushed to the buffer
    if (packet_size > bytes_to_send) {
        bytes_to_send = 0;
    } else {
        bytes_to_send -= packet_size;
    }

    // move our transmit buffer head forward
    write_head += packet_size;

    return status;
}

Next, we’ll modify the receive callback so that it fills a buffer with a specified amount of verifiable data, and starts sending it back to the host. 

err_t recv_callback(void *arg, struct tcp_pcb *tpcb, struct pbuf *p,
                    err_t err) 
{ 
    /* do not read the packet if we are not in ESTABLISHED state */
    if (!p) {
        tcp_close(tpcb);
        tcp_recv(tpcb, NULL);
        return ERR_OK;
    }

    xil_printf("entered recv_callback\r\n");
    /* indicate that the packet has been received */
    tcp_recved(tpcb, p->len);

    // convert the first two bytes of the received packet's payload
    // to a 16-bit unsigned integer
    bytes_to_send = *((u16_t*)(p->payload)); 

    xil_printf("Client asked for %d bytes\r\n", bytes_to_send);
    // load the buffer 
    for (int i = 0; i < bytes_to_send; i++) {
        test_buf[i]= (u8_t)(i & 0xFF);
    }
    write_head = test_buf;
    if (bytes_to_send > 0) {
        err = push_data(tpcb)
    } // else, nothing to send 

    /* free the received pbuf */ 
    pbuf_free(p); 

    return err; 
} 

Lastly, we’re going to add the new sent_callback function, which will continue sending data from the buffer until it empties out. 

err_t sent_callback(void *arg, struct tcp_pcb *tpcb, u16_t len) {
    // log to USBUART for debugging purposes
    xil_printf("entered sent_callback\r\n");
    xil_printf("    bytes_to_send = %d\r\n", bytes_to_send);
    xil_printf("    len = %d\r\n", len);
    xil_printf("    free space = %d\r\n", tcp_sndbuf(tpcb));

    // if all bytes have been sent, we're done 
    if (bytes_to_send <= 0)
        return ERR_OK;

    return push_data(tpcb);
} 

Note that this code is not robust and some situations could cause problems, like if another block of data was requested while the server was in the middle of sending a previous one. That said, it could easily be modified further so that the server can accept some arbitrary command, take action based on what’s in that command, and return requested data or some other response to the client. 

Python socket example 

Next, we’re going to set up a Python client that can connect to the server, request some data, and check it to see that everything made it through as expected. 

The socket module implements an easy-to-use API for TCP connections and is included in a basic Python installation by default. The following chunk of code will connect to the Zynq server after it is running on the board, request some bytes, and then check that they match the expected. Make sure to run the Zynq application before the script. 

import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# connect to the server; enter your server's IP and port here, as
#  printed by the board's serial interface
client.connect(("10.0.0.128", 7))
# specify the number of bytes we're requesting from the server
# change this as desired, several values up to 50k have been tested
# for much more, more than two bytes would need to be used to
#  specify length
num_bytes = 500

# arbitrary packet size, max number of bytes we'll receive at once
packet_size = 256

# send two bytes representing num_bytes to request that many bytes
#  in response
# note: little endian is important, requirement for Zynq-7000 to
#       easily translate the sent number to an int without reordering
print(f"requesting {num_bytes.to_bytes(2, 'little')} bytes")
client.send(num_bytes.to_bytes(2, 'little'))

# loop while calling recv to receive data from the client until the
# expected number of bytes has been successfully transferred
received = 0
errors = 0
while received < num_bytes:
    data = client.recv(packet_size)
    for d in range(len(data)):
        expected_value = (received + d) & 0xff
        if data[d]!= expected_value: # validate data
            print(f"Error, data[{d}] ({data[d]}) != {expected_value}")
            errors += 1
    received += len(data)
    print(f"Received {received} bytes total, {len(data)} in this recv")
if errors == 0:
    print("All data received matched the expected values!")
else:
    print(f"{errors} errors")

Results 

Let’s see what all of this looks like in action. In this video, an Eclypse has already been connected to a host computer via USB and connected to the router via Ethernet. We’ll run the modified echo server application in Vitis, check the IP address the board picks up in an already-connected instance of Tera Term, then run the python script to transfer and check some data: 

Note: This code was tested using Vitis 2023.1.  

In this post, we’ve explored how to extend the capabilities of an LWIP echo server for Zynq-7000 boards, going beyond the standard echoing of received data. By programmatically sending custom data back to the client, we’ve demonstrated the potential for more advanced data manipulation within the server software application. While we’ve showcased a basic implementation in this article, it’s important to note that the possibilities are vast. You can easily adapt this approach to handle data from various sources, such as sensors or other FPGA-connected devices. The power of Zynq presets enables you to dive into this process swiftly, utilizing your existing hardware platform. As you embark on your data transmission journey, remember that this new capability opens doors to innovative applications and solutions. Whether you’re refining your existing systems or jumping into entirely new projects, the flexibility of Zynq-based Ethernet communication paves the way for exciting developments in the world of embedded systems and networking. 

0

0

Be the 1st to vote.