I am your “Personal Technology Advisor”. Much like your own accountant, lawyer, or anyone else to whom you would go for targeted, personalized advice, I am your source for coaching and direction on the business and technology issues that impact your company and contribute to your own professional growth. I'm here to offer seasoned guidance when you’re venturing into unfamiliar territory, solid expertise when circumstances require quick action, and a discreet and unbiased ear when you need a reality check. My technology and leadership career spans 27 years in the fields of entertainment, advertising, digital media, legal services and retail, including fulfillment logistics and e-commerce. In progressive management roles, he has guided technology operations, software development and interactive media for small, privately-held firms, mid-sized enterprises (Z Gallerie, Hot Topic, Newegg) and multi-national Fortune 500 companies (Prudential Insurance, The Walt Disney Company).
Good answers so far, so I'll echo a few things and refute some others. Having worked on both the agency and client sides, I can say:
1. Consider portfolio and process with equal weight. Absolutely, past work is an indicator of creative capability, but without process, you may be entering a world of frustration and cost overages. Pay special attention to how they propose to communicate with you and to the kinds of questions they ask. Even if you've worked with agencies before, go into your investigation with them as if you haven't - consider how they guide you and explain their perceived roles and responsibilities. You do yourself no favors by playing know-it-all.
2. Do not hire an agency without talking to two reference customers, asking each specifically about the agency's process.
3. Contrary to some other answers here, place little weight on whether they've done work in your industry. A good agency, working with you to understand your product and its value proposition, can do branding in any industry. If you consider it at all, consider it lightly - it's insulting to the agency and limiting to you and your search.
4. Frankly, the biggest factor for success in working with any agency is you knowing what you want at the end of the project. What are the success factors? Conversely, not knowing your own goals is the surest way to make any agency project fail. They can help you tease that out and solidify it through a good creative brief process, but it's not the agency's job to tell you what you want to achieve for your brand.