These days it is rare to see someone NOT on their phone, either browsing the internet or using apps. This is an important reason why you should ensure your website is mobile optimised.
Mobile optimisation allows people to view your website on any device, making the site responsive to differing screen sizes. If you look at a non-optmimised site on a mobile, you might see that the text displays really small, and contact buttons are difficult to click on. These are things you want to minimise on your mobile site in order to keep a good visitor count.
Most web design agencies can advise you on how best to optimise your site for mobile devices, and can often do a quick analysis of your current setup to figure out how best to fix your issues. Some of the things they will do will be to use code to make the website automatically reformat for mobile screens and speed up the load time, which is a key factor in visitor retention.