PS Vita button issue

Since there really isn't any way for a button to stop working due to software/firmware issues, it definitely sounds like a hardware problem.

Fortunately, the PS Vita is much easier to perform one's own repairs on than most compact electronic devices. The internal design is surprisingly modular which allows for easy replacement of the buttons, sticks, and the left/right printed circuit board (PCB) modules which actually register the input of a button press. Replacement buttons and circuit boards can be found at a number of online retailers and are relatively inexpensive (the buttons are under $10 and the boards range from $15-$25 on Amazon). You'll want to make sure to order the correct parts for your Vita; there are slight part differences between the 1000 and 2000 models, as well as differences whether or not it's a Wi-Fi or 3G model. However the part prices and replacement procedures are basically the same.

Unfortunately the compact design makes troubleshooting the individual connections a pain, but because the parts are so cheap it's no big deal to go ahead and order the buttons and the right side PCB module to save yourself the headache. Of course you'll also need some small precision screwdrivers commonly used for computer & electronic repair if you don't already own a set. There are plenty of written instructions with photos and diagrams online to explain the procedure, and a number of YouTube videos demonstrating the process. You only have to deal with screws and ribbon connectors; no soldering is needed.

Of course I only recommend doing this if you have some basic knowledge or experience replacing parts in high-tech compact devices, otherwise you're better off taking it somewhere that repairs game consoles. The ribbon connectors and the sockets they plug in to are especially delicate so if you aren't gentle or try to force it the wrong way you can easily damage something else or harm your new replacement parts. A shop can make mistakes too, but at least they'd have to fix their screw-up at their own expense.