webrtc_streamer doesn't have attribute tobytes() to convert the captured image to img_bytes and make an API call

[sandeshkatakam]

I am trying to build a face recognition system and I chose streamlit as my frontend UI to make API requests. I am using webrtc_streamer class to capture image live camera feed and when I try to capture a frame and convert it into numpy array using cv2.imencode() or even when I try to convert it into image bytes using tobytes() function I am not able to do it because webrtc-streamer class doesn't have tobytes() attribute. Is there any fix such that my image capture from webrtc_streamer will be a numpy array class so that I can make the API call to flask server possible?

def login_page():
    st.title('User Login with Face ID')
    st.write('Use the camera feed below to capture your face image for login.')

    
     webrtc_streamer(
        key="streamer",
        video_frame_callback=transform,
        sendback_audio=False
    )
    
    if st.button('Login'):
        # Perform face recognition and login process here
        # Convert the image to bytes
        _, buffer = cv2.imencode('.jpg', image_capture.frame)
        image_bytes = buffer.tobytes()
        

        # image_bytes = image_capture.frame.tobytes()

        # Send the image to the recognize_face API endpoint
        # Convert the image to JPEG format
        # image_jpg = cv2.imencode('.jpg', image_capture.frame)[1].tobytes()
        
        # Send the image to the recognize_face API endpoint
        response = requests.post('http://localhost:5000/recognize_face', data = image_bytes)

        # Get the result from the response
        result = response.json()

There is also an alternative method I have tried

class ImageCapture(VideoTransformerBase):
    def __init__(self):
        self.frame = None
    
    def transform(self, frame):
        img = frame.to_ndarray(format="bgr24")
        return frame
        
def login_page():
    st.title('User Login with Face ID')
    st.write('Use the camera feed below to capture your face image for login.')

    image_capture = ImageCapture()
    webrtc_streamer(
        key="streamer",
        video_frame_callback=transform,
        sendback_audio=False
    )
    
    if st.button('Login'):
        # Perform face recognition and login process here        

        # image_bytes = image_capture.frame.tobytes()
        
        # Send the image to the recognize_face API endpoint
        response = requests.post('http://localhost:5000/recognize_face', data = image_bytes)

        # Get the result from the response
        result = response.json()

Both the methods are giving the following errors:

• First method Error:
  cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in function 'imencode'
• Second method Error:
  AttributeError: 'WebRtcStreamerContext' object has no attribute 'tobytes'

[whitphx]

• I think you should walk through the tutorial in the README
• In the first place, looks like you can use Streamlit's st.camera_input() for that purpose where you want a single image input to make the request.