The security concerns should not be your main focus. It's getting the thing to work. I have seen countless times agencies overpromise and underdeliver. The contract needs to be structured in a way to prevent them wasting months of your time promising a product they can not deliver. The other main point to remember, is that likely you will need to hire full time, in house, developers to transition the software if you start to get traction. Most of the times agencies build code which does not scale well. Usually this is due to a tradeoff between speed of development (getting the software to you) and efficient, well architected code.

My advice, whatever it's worth is if you can afford it, build it in house from the start. You will most likely end up rewriting the code once you bring in a CTO. This has happened to almost everyone I know who has worked with an agency.
If you can't afford it, get as many references as you can from completed projects, structure the contract in a way to prevent garbage code, and think through your software from the start to prevent feature creep, agencies hate this, and will make your final code less efficient if you keep springing new features as the build progresses.