News Stay informed about the latest enterprise technology news and product updates.

Performance management tools ranked, vendor rankled

Ventana Research ranked performance management technologies by how well they support business processes, yet some vendors were notably absent from its scorecard.

A recent study of performance management vendors evaluates and ranks many of the technologies on the market --...

but a few key players are missing.

San Mateo, Calif.-based analyst firm Ventana Research recently released its 2006 Performance Management Vendor and Product Scorecard -- a study it's been doing semi-annually for five years -- which rates vendors on how well their technology supports overall business processes of performance management. Some performance management vendors, such as Hyperion, SAP and Microsoft, didn't meet Ventana's deadlines to answer questions and thus didn't make this study, according to Mark Smith, chief executive officer of Ventana. But there may be more to the story. Others declined to participate, and one vendor questioned Ventana's methodology.

Ventana's performance management tool rankings
Vendor Overall
Extensity 1
SymphonyRPM 2
Cognos 3
Business Objects 4
Oracle 5
Epistemic 6
Information Builders 7

Source: Ventana Research 2006 Performance Management Vendor and Product Scorecard
Of the 14 vendors included in the final study, Extensity RPC (formally Geac), SymphonyRPM, and Cognos Inc. had the top three overall scores, respectively. Scores were also broken out into three sub-categories.

"We looked at enabling sets of functionality that can support the three steps of performance management, which are to align, optimize and understand performance," Smith said. "It's not a subjective report -- [vendors] have to actually demonstrate that they have these types of capabilities. They either have the functionality or they don't have the functionality."

According to the firm's methodology, companies must be able to understand how past actions affected performance, align workforce plans and actions with corporate goals, and optimize future performance through collaboration and planning, Smith explained. Different technologies have different sweet spots.

The "understand" step is well understood by many vendors, Smith said. Researchers looked at functions for modeling, discovery and measurement of performance, as well as ongoing monitoring capabilities. This resulted in a three-way tie among Actuate, Extensity and SymphonyRPM for the No. 1 spot, and a four-way tie among Cognos, Epistemic, Oracle and Information Builders for the No. 2 ranking.

Technology has really evolved in the "align" category since last year, Smith said. Products are better at aligning the workforce with strategic goals, communicating and coordinating targets, scoring performance, and enabling appropriate rewards and incentives. The leaders in this category were Cognos, Extensity and Softscape in the first three spots, followed by SymphonyRPM, ranked fourth, and Pilot Software, fifth.

Many vendors still have work to do on functions for optimizing performance and preparing for future actions and decisions to achieve corporate goals, Smith said. Extensity and Symphony tied to lead the category by a wide margin, with Business Objects and Epistemic distant followers. Cognos, a leader in the other two categories, ranked seventh in this area. Vendors will beef up this area of their offerings this year, Smith predicted.

Software buyers can use the study results to compare technology functionality to specific needs, Smith said. When it comes to buying performance management tools, rank is less important than requirements, he noted.

"Some companies don't need all the functionality," Smith said. "When we use this tool with corporations, we ask them, 'What capabilities do you have today? Where do you want to go? What do you want to improve upon?' That gap is what you need to focus on and find technology that fits."

Dissension in the ranks?

The report does not rank all performance management vendors, however. The study notes that IBM and Computer Associates chose not to participate and that Hummingbird, Lawson, Longview, Noetix, Outlooksoft, Panorama and QlikTech did not respond to Ventana's request for participation. Further, it said that Arcplan, Corporate Radar, Hyperion, Microsoft and SAP "participated, but did not complete the auditing and validation according to the schedule."

SAP completed its submissions but didn't validate its pricing by providing the required screen shot of the product price list, Smith said. Hyperion didn't complete the submission process in time, he added. The recent announcement about Microsoft Office PerformancePoint Server 2007 came after research was completed and wouldn't have been included anyway because the product isn't released yet. Microsoft also missed some key deadlines for the evaluation of its existing performance management options, Smith said.

For more on performance management

Read an expert's advice about how to approach performance management

Learn why Gartner sees performance management and business intelligence converging

Microsoft and Hyperion are likely to complete the required steps soon. If they do, Smith said, they will be included in an update to the study later this year. But Hyperion apparently isn't so sure, despite ranking 10th in Ventana's November 2005 study and being named as a leader in Gartner's October 2005 Magic Quadrant for performance management.

"Hyperion did not participate in this Ventana study due to our concerns about whether its vendor and product rankings serve the best interest of software buyers. We question whether the methodology and results take into account the unique requirements of companies considering performance management solutions or reflect the full capability and flexibility of the solutions that vendors can provide to address their customers' needs. Further, we don't believe any analyst firm is capable of ensuring the accuracy and consistency of survey responses gathered from vendors," a Hyperion spokesman said in an emailed statement.

That's news to Ventana, Smith said. Hyperion, which has participated in the performance management study in years past, said it wanted to participate again this year and simply missed the deadlines for submission. Hyperion has not contacted Ventana to discuss the firm's methodology, Smith added. He stands behind Ventana's assessment methodology, but said he's willing to discuss it.

"This [study] has been going on for five years, and this is the first time I've heard anything like this," Smith said. "[Hyperion is] putting out an anti-position based on their lack of ability to complete the study in the time period. They have the right, like any company, to provide feedback or input and have never voiced anything back to us or given me any indication about what [they think] is wrong with the methodology."

Dig Deeper on Performance management software

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.