The old steam.exe should be left renamed to Steam.exe.old in your steam folder, simply remove the.old extension part and run it with avast disabled temporarily and it'll re-update and correctly get the new Steam.exe. Finally, you should verify the integrity of your game files. Here is how you can do that: Restart your PC. Go to Library. Right-click on your game. Click Properties. Proceed to Local files. Click the Verify integrity of game files button. Please be patient – the verification process may take a while.
Update: This problem has been addressed by Microsoft and was fixed in the big November Update of Windows 10. Shortcuts created by Steam should work as expected now.
Edit: This is a useful workaround that you might want to use until Windows Search gets updated to display .url files in the start menu search. Some might consider this workaround a better solution in general. Thanks a lot for taking the time /u/Matzurai
This is a clean install of Windows 10 Home(64 bit), both Startmenu locations are indexed, I triple checked that the shortcuts are inside an indexed folder and double checked that indexing had actually finished. Moving the Shortcuts to a different, also indexed, location didn't make a difference.
Windows search finds the Steam shortcut itself, and has no issues with other Internet Shortcuts(it does, I was stupid), it just doesn't find the Steam ones.
Does anyone have a solution for the problem, or can at least confirm that I'm not the only one suffering from it?