Get Help
Skip to Main Content

Accelerating Academic Research with AI - GAIL Conference '24

Perplexity

Activity: Evaluating and Assessing the Generative AI Tool, Perplexity

Rating Scale: Use a 1 to 4 rating scale to assess various aspects of generative AI tools. Assign a score from 1 (lowest) to 4 (highest) based on the criteria outlined below:

  1. Relevance: Evaluate the tool's relevance to your specific needs and objectives. Consider whether the tool aligns with your goals and addresses your challenges effectively.

  2. Ease of Use: Assess the tool's usability and user experience. Evaluate how easy it is to navigate the tool, perform tasks, and access its features. Consider factors such as intuitiveness, accessibility of functionalities, and clarity of instructions.

  3. Assessing DEAI (Diversity, Equity, Accessibility, and Inclusion): Evaluate the tool's performance in promoting diversity, equity, accessibility, and inclusion. Consider whether the tool incorporates diverse perspectives, mitigates biases, and ensures equitable outcomes for all users, regardless of their background or abilities.

  4. Currency: Examine the tool's currency in terms of its technology, data, and methodologies. Assess whether the tool utilizes the latest advancements in AI research, incorporates up-to-date datasets, and follows current best practices in generative modeling.

  5. Transparency & Accuracy: Evaluating the tool's transparency and accuracy in its operations and decision-making process. Assess whether the tool provides clear insights into how it generates outputs, handles data, and makes predictions, as well as whether it consistently delivers precise and correct information. Look for transparency in the algorithms, data source, model limitations, and potential biases that may affect its performance.   

Usage Instructions:

  • Review each criterion carefully and consider its relevance to your evaluation.
  • Assign a score from 1 to 4 for each criterion based on your assessment of the tool's performance.
  • Use additional comments or notes to provide context and justify your ratings.
  • Aggregate the scores to obtain an overall assessment of the tool's strengths and weaknesses.

Conclusion: By utilizing the REACT framework, you can systematically evaluate and assess generative AI tools, enabling informed decision-making and facilitating the adoption of tools that best meet your needs while aligning with your values and priorities.