Our software represents a groundbreaking advancement in addressing the communication challenges faced by sign language users. By leveraging cutting-edge computer vision and machine learning technologies, our system adeptly captures sign language gestures in real-time with unparalleled precision, seamlessly translating them into easily understandable text and audio formats. Addressing the urgent need for enhanced communication accessibility, our software offers a revolutionary solution that fundamentally transforms how sign language users engage with the world. What truly distinguishes our platform is its exceptional accuracy, versatility, and user-friendliness, facilitated by sophisticated machine learning algorithms meticulously trained on extensive datasets. Beyond its core functionalities, our software boasts an array of features, including support for various sign languages and user-friendly tutorials tailored for beginners, thereby promoting inclusivity and accessibility while democratizing access to sign language proficiency. Furthermore, with support for multiple languages, our software enables users to comprehend sign language gestures in their preferred language, dismantling linguistic barriers and fostering global communication inclusivity. Committed to ongoing research and development, we ensure that our software remains at the forefront of innovation. In essence, our software transcends mere communication facilitation – it empowers individuals, fosters inclusivity, and envisions a world where effortless communication is accessible to all, irrespective of language or ability.
- Sreehari S
- Induchoodan VS
- Sofiya B
- Harigovind P Nair
This code is a Python program that aims to detect and recognize hand signs using computer vision and machine learning techniques.
- Importing Libraries: The code starts by importing necessary libraries such as OpenCV (
cv2), MediaPipe (mediapipe), NumPy (numpy), Tkinter (tkinter), and others. These libraries provide functionalities for image processing, machine learning, graphical user interface (GUI) creation, etc. - Sign-in Window (GUI): The
signin()function defines a graphical user interface (GUI) for user registration. It uses Tkinter to create a window with input fields for email, username, password, and confirm password. Users can register by entering their information, and upon successful registration, the data is stored in a MySQL database. - Data Collection: The
collect_imgs()function collects hand sign images from the user. It captures images from the webcam, saves them in a directory (./data), and organizes them into folders corresponding to different hand signs. - Dataset Creation: The
create_dataset()function processes the collected images to create a dataset for training the machine learning model. It uses MediaPipe to extract hand landmarks from the images and stores the extracted features along with corresponding labels (hand signs) in a pickle file (data.pickle). - Model Training: The
train_classifier()function trains a machine learning model (Random Forest Classifier) using the dataset created in the previous step. It splits the dataset into training and testing sets, trains the model on the training data, and evaluates its performance using the testing data. The trained model is saved in a pickle file (model.p). - Inference (Detection): The
inference_classifier()and 'audiowork()' functions implement real-time hand sign detection using the trained model. It creates a GUI application using Tkinter where users can start the detection process. The application accesses the webcam, captures frames, detects hand signs in the frames, and predicts the corresponding hand sign using the trained model. Detected hand signs are displayed on the screen along with the webcam feed. - Additional Functionality: The
newsigns()function provides options for users to import hand sign images, confirm hand sign registration, and check detected hand signs. It interacts with other modules (collect_imgs.py,create_dataset.py,train_classifier.py,inference_classifier.py) to perform these tasks.
- openCV: 4.9.0.80
- mediaPipe: 0.10.11
- pickle: 0.3.0
- scikit-learn: 1.4.2
- tkinter: 0.1
- numPy: 1.26.4
- random: 1.0.0
- pyMySQL: 1.1.0
- cryptography: 42.0.5
- pyttsx3: 2.90
- Setting up the Environment:
- I started by ensuring that Python is installed on my computer. If not, I downloaded and installed Python from the official website.
- Library Installation:
- Next, I used pip, Python's package manager, to install the required libraries listed in the program. This command installs all the necessary libraries so that the program can run smoothly.
- pip install opencv-python mediapipe scikit-learn pillow pymysql cryptography
- Database Configuration:
- Since the program interacts with a MySQL database, I made sure to have MySQL installed and running on my system. Then, I configured the program to connect to my MySQL database by providing the hostname, username, and password in the code.
- File Paths:
- I checked that all the image files and resources such as (icon.png, bg.png, openeye.png, closeye.png, etc.) referenced in the program exist in the specified locations. If needed, I adjusted the file paths in the code to match my project's directory structure.
- GUI Configuration:
- The program uses Tkinter for the graphical user interface. I reviewed the code to understand how the GUI elements are structured and customized them according to my preferences. This involved adjusting the layout, styling, and functionality of the GUI.
- Testing and Debugging:
- After configuring the program, I thoroughly tested it to ensure that all components work as expected. This included testing user interactions, database operations, and any other functionalities. If I encountered any errors during testing, I debugged them and made necessary corrections.
- Running the Program:
- Once everything was configured and tested successfully, I ran the program by executing the main Python file. This launched the program, and I was able to interact with it using the graphical user interface.
- Additional Customization:
- Depending on my specific requirements, I made additional customizations to the program. This could include adding new features, modifying existing functionalities, or integrating with other systems or services.
To run this program:
- Ensure you have all the necessary dependencies installed. You can install them using pip if you haven't already: pip install opencv-python mediapipe scikit-learn pymysql pyttsx3 cryptography
- Place all your image data for training in the
datadirectory. - Uncomment the lines
#collect_imgs(),#create_dataset(), and#train_classifier()if you need to collect new images, create a dataset, and train the classifier respectively. These steps are optional if you already have a trained model and dataset. - Ensure you have the required image files (
icon.png,bg.png,bdb.png,openeye.png,closeye.png,openeye1.png,closeye1.png) in the same directory as your Python script. - Run the Python script. This will open a GUI window where you can sign in or register. Once signed in, you can start hand sign detection and explore other functionalities like adding, learning, and translating hand signs. Depending on the options selected during registration or sign-in, you may also get audio output for detected hand signs.
Remember to handle the database setup separately according to your environment. Also, make sure to adjust any file paths or configurations as needed for your specific setup.