This 17-year-old has built an AI driven software for visually impaired - Siddhanth Lohia
Siddhanth Lohia is just 17 years old and is studying in 11th grade at the SevenOaks boarding school in London. Two years ago after watching a video on ChessBase India, where we had Aryan Joshi and Soundarya Pradhan, two of India's top visually impaired chess players, talk about the issues with accessibility, Siddhanth made up his mind to do something special. He learnt about AI from the workshop that he did when he was 15. Putting that to use, he has come up with something truly special. It is a software which a visually impaired chess player can install on his machine and then just press a button and it can give him the position of the chess pieces on the board that is present on the screen. It is an extremely important innovation and can really help the visually impaired chess players take their game to the next level.
A groundbreaking Software for visually impaired chess players!
IM Sagar Shah (SS): I'm joined here by a very young talented chess player! However, I am doing this interview with him not because of his chess achievements, which I am sure are many, but because he's made something truly special for the world of chess. Siddhanth, great to have you here!
Siddhanth Lohia (SL): Hi sir, it's amazing to be here!
SS: Sidhant, you have made something known as AI-Chess Project. You have a website called chessforall.in. Could you tell us what is this project all about, and which problem does it solve?
SL: Sure sir. As you said, I've been playing chess since I was 5 years old, and I've played chess across the world. I've represented India, and along with that I also have a passion for technology and computer science. So I thought, maybe I can combine these two passions of mine and create a project which can also give back to the sport that's given me so much. So, I've made a project that involves artificial intelligence and neural networks to help visually impaired chess players access chess resources online.
SS: So you made use of AI to help the visually impaired chess community? I am very interested to know how it works, because I have spent quite some time with visually impaired chess players, and I know that they have trouble accessing chess content. So, which of the issues did you did you solve through what you made?
SL: I actually came across this problem from one of your videos. I haven't told you this before, but I came across this video - I think you were with Soundarya and Aryan in a hotel room. They were showing you the issues they were facing with accessing chess resources online.
This is when it just sparked a thought in my brain, and I thought maybe I could do something to work on this, and to help them access these chess resources online. I had an interest with AI, image recognition and neural networks. That's when I set off to work on this project, and this was two years ago. Now it's finally been packaged! As of now, the chess software in its first version helps them access all online chess websites and all online chess resources. So, they can access any websites like chesscom and lichess, ChessBase tactics, softwares like fritz, ChessBase and even your videos. If they want to watch YouTube videos and they want to see what's going on in your livestreams, like the world championship match right now, they can have access to that. They can know what's going on on the chessboard!
SS: Wow! Okay, I am very interested to see all of this – how it pans out in real life. We will try to access the website, also I want to know what inspired you to do it. How was your journey, and how did you begin? Because if someone went to University, studied engineering, then studied AI and stuff, I could understand them doing it. But you are just 17, you just finished school, so from where did you learn all of this? I want to know all of that. But first, shall we go and check how it works?
SL: Sure sir, let's do it!
This is the page for the AI Chess Project! You can download the software through this page, and also find instructions on installing and using it.
SR: After downloading and installing the software, it keeps working in the background.We've kept it running in the background, because for the visually impaired, it makes it easier for them to manage the processes going on. It also always works in the background, and only activates when you click a shortcut and when they want to use the chessboard.
SS:One of the things which I realized and I think you have found, is that when a game starts from the first move, a visually impaired chess player can follow it, because they have the position and then they can make the moves. But let's say when you access a chess position like this in the ChessBase tactics portal, then it's very difficult for them to know what the position is there on the screen, correct?
SL: Right, there's no way for their screen readers to access a chessboard, because it's an image. There's no way for that screen reader to access this, so that's where my software comes in.
SS: All the visually impaired chess players usually have a screen reader, which reads out anything that's written on the screen. So if it's written here "Black to move and win", it would read it out. But the problem is, it wouldn't read out the coordinates of the pieces. So, what is the step I need to do now?
SL: After speaking to a lot of visually impaired chess players in this process, they said most of the programs they use are based on the shortcuts. So, to run this program you need to execute the keyboard combination Alt+3.
SL: Their screen readers read it out to them at extremely fast speeds, so this entire process happens to them within a second or two, and they know the chess position immediately. Because we're working on the first version of the software, what our software does now is it just takes a screenshot of your screen, and then it runs that screenshot through the neural network. Sometimes if the chess software or the chess board is too small, then it's not able to capture it. So, you just need to ensure that the chessboard is visible, and it's big on the screen.
SL: What I found in ChessBase.in, if you go to one of your articles like other one for this world championship match, you have diagrams that start in the middle of the screen. So, if a visually impaired chess player wants to access this website, they wouldn't know after which position this you know this diagram or the notation starts. Now if you zoom in and press Alt+3, they'll be able to see everything.
SS: One of the things which is again very interesting, and I guess you have thought of many applications of this. But there can be many more, which you may not even have foreseen! One of them as we discussed is, let's say there is a chess live stream going on in in twitch or twitter, they can follow it!
SL: I think that was a big thing. I was speaking to a few visually impaired chess players - I spoke with Darpan Inani,I spoke with Kishan Gangooli.When I told them about the application for live streams, that was the big thing and they were really excited about that. This is something that was never accessible to them before ,so this is just this opens up a whole lot of possibilities.
SS: Someone who wants to improve their blindfold chess, can also use this, right? You can open a position, then you can do this Alt+3, and then read out the notation and try to visualize the position in your head.
SL: Yes sir, for sure. You can also use this even for sighted people, if you ever want the notation of any chess material online for any reason, you can just use this. If you want to send a position to someone, you can do it!
SS: I think you have also designed a very small GUI, if I right click on the chessboard icon on the taskbar.
SL: Yeah, this is for anyone who wants to know how it works. That's why we have a small GUI, it's not needed for the original plan.
SS: How and when did this idea begin? You said it was 2 years ago.
SL: Actually like I mentioned, I had a passion for technology and computer science since I was a kid. I did a summer camp two years ago on artificial intelligence and neural networks. After I did that summer camp, is when I got a lot more knowledge about this,and I wanted to start a project. That's also when I came across the video on your channel, where Aryan and Soundarya explain the difficulties they were facing with these chess resources online! I thought I can combine the two - neural networks and image recognition, and maybe create a software that can help them access this.
Obviously, we found existing neural networks that had all these different type of chess pieces. Because, the software works with all the different designs of the chessboard, chess pieces with different colors. as we used all these different websites. We had to make sure it encompasses all of that, and after that I created a basic software GUI. Then I packaged it into an application. Obviously, this entire process took a long time because I was just 15 at the time, so I had to take some help from a mentor that was studying at Carnegie Mellon. He just guided me with the project, because it's my first time working with artificial intelligence and an application like this, but the majority of the work was done by me.
SS: This is brilliant! So what you're saying is that the AI is basically required, so that it can identify all different types of pieces. Because you can't possibly sample all different types of looking pieces, right? The AI recognizes this, like this has a cross somewhere, this must be the king.
SL: Exactly, that's how it recognizes any board, any piece in the chessboard. It identifies the form, then it gives the coordinates. What the neural network does is, it just takes the software and it converts it into a FEN code. Another program converts the FEN code into text, and then I had to make the program such that the FEN code comes on a notepad file, which then pops up on the screen immediately., so that the visually impaired people can read it and it's accessible. This entire process is instantaneous, so all they have to do is just go to the software, just click the keybind (Alt+3), and immediately they have the notepad file with the coordinates of the pieces, and they can read everything.
SS: Brilliant. Do you have any future plans of making it even further better, or making more features around it?
SL: Yes sir. So what I actually want to work on next, is obviously the issue where it doesn't capture the chessboard unless it's enlarged on the screen. So maybe working on that, and then I think the biggest thing would also be to make it accessible with online PDFs and just book PDFs. Because right now, the reason our software doesn't work with those softwares, is because in chess books, the background on black squares with the pieces is very different. It's only distinguishable by a human eye, because on the software there's a brown background, a blue background, or a green background. There you can easily differentiate between the piece and the background, but on a PDF or on a book. it's very hard to do that. That's definitely my next project, the next version, that's something I want to work on.
SS: I think you would do it. Because you've already cracked this, that shouldn't be so tough. What do you do Siddhant, you're studying in London right?
SL: Right sir, this is my first year studying boarding school in London. I'm in 11th grade right now, in Seven Oaks School.
SS: What is what is your aim in general? You are also a strong player I believe, you gained a lot of rating recently. I think you were around 1600 or so, and you gained 300 Elo points in the last few years that you were playing chess actively.
SL: The lockdown is when I stopped playing chess. My FIDE rating is a bit low compared to my strength, I would say. That's when these interests of mine picked up. But now that I've gone back to the UK, I've picked up chess there as well. I have a strong ECF rating, and we just won the National Regional chess finals in my school. I started a chess team in my school, and this is the first event we're competing in. We've already won the chess finals, and in a week (19th April), I'm going back to London and we are playing in the National Chess challenge finals against the best teams in the UK at Imperial College on 19th April. I've started playing chess properly!
SS: You went to a new place, a new country, but I think your affiliation with chess makes you feel at home, yes?
SL: Yeah obviously sir, always. There's been a huge pickup of chess, so I teach chess in my school and my boarding house. I've started a chess ladder, and I promoted people to join the chess club in school, which is actually taught by a very famous Grandmaster John Emms. So, John Emms teaches the chess club in my school, and I've promoted a lot of my friends to join the chess club. I actually helped them form a team, which competed in a tournament as well.
SS: Siddhanth, you're doing so many things at such a young age for chess, and it's absolutely great to see that. Finally, as I end this, a lot of people will definitely watch this and get to know. Who are the people you would like to thank for helping you to get this project done?
SL: Definitely my parents, they've supported me throughout this. I would thank my mentor Ishan, who was studying Machine learning at Carnegie Mellon. There was another one of my dad's friends, Sachin uncle, who I always went to if I had any technological doubts. They've definitely helped me a lot.
SS: Brilliant! I am actually going to keep a track of all that you are doing, Siddhanth.With chess technology growing all the time, and also the AI becoming better, I guess there will be so many things that will come in to the sport. You might be the one who will be at the Forefront of all of this! Wishing you all the best.
SL: Thank you so much sir!
Important links
Chess for all Official website