AI Evaluations Research Associate

Job Type:

Full-time or part-time

Overvie w

RANDs Technology and Security Policy Center (TASP) is seeking mission-driven AI Evaluations Research Associates to develop and execute research projects and engineering efforts within our AI Capability Evaluations (ACE) team.

RAND's reputation for excellence is built on our commitment to high-quality, rigorous analysis and objectivity. TASP is at the forefront of research and implementation regarding the impact of high-consequence, dual-use technologiessuch as artificial intelligence and biotechnologyon global competition and security. Our research has been used by the White House, U.S. government departments and agencies , the EU and UK governments, and industry leaders, among others. Our alumni have gone on to important roles at the NSC, Commerce, DOD, Congress, Google DeepMind, OpenAI, EU AI Office, UK AISI, other key think tanks, and founding mission-driven tech initiatives .

ACE develops and conducts evaluations of national security relevant capabilities of frontier AI systems, with a current focus on the intersection of large language models (LLMs) and AI agents with biological risk. Were hiring for people with research scien ce and /or research engineering skills to play a key role in work that assists public policymakers at all levels in strengthen ing national security and mitigat ing catastrophic risks enabled by AI systems. They will work on complex problems at the intersection of AI and national security where technical details matter and will contribute to multidisciplinary project teams that include biosecurity experts, machine learning engineers, and policy researchers.

This position is initially structured as a focused 1-year appointment to create the urgency needed to drive ambitious change in this rapidly evolving field. Every day of your tenure will count toward that goal. The appointment may be renewed for up to a total of 3 years , with options for longer-term employment at RAND thereafter. Full-time and part-time (at least 20 hours per week) schedules will be considered, but with a strong preference for full-time.

Responsibilities

Given the breadth of valuable work our team could do, there is some ability to align responsibilities with an individuals skills, interests, and career goals , including in terms of the balance of research scientist - versus research engineer -style responsibilities . Responsibilities may include but are not limited to:

• Contribute to developing concrete threat models for high-consequence risks AI risks, working with internal and external partners

• Contribute to d esign ing and execut ing rigorous, objective evaluations of AI capabilities relevant to key bottlenecks within those threat models

• Contribute to d evelop ing and maintain ing the technical infrastructure required to support this research, working with relevant internal and external IT stakeholders

• Develop and maintain code for fundamental evaluation components that can be used ac ross research efforts ( e.g. prompting, auto mated grading , statistical analysis)

• Keep up to date with the latest advance s in AI evaluation engineering and the science of evaluations to continually im prove the rigor and efficiency of our evaluations

• Contribute to c ommunicat ing research results to policymakers and other key stakeholders at all levels through written products and oral presentations

• Contribute to setting strategic and research priorities, with an emphasis on the policy impact of evaluations

Qualifications
All research positions at RAND require excellent analytic skills; the ability to communicate clearly and effectively in English, both orally and in writing; the ability to work effectively as a member of a multi-disciplinary team; and a strong commitment to RAND's core values of quality and objectivity.

Other r equired qualifications :

• Strong interest in understanding and addressing potential national security risks related to autonomy or high-consequence misuse of LLMs and AI agents , and in AI capability evaluations as a route to impact

• P roficiency in Python

• Familiarity with technical aspects of AI systems and related technologies, such as machine learning, computational infrastructure, or information security

Preferred but not required :

• Experience with evaluations and evaluation frameworks for LLM s and AI agent s ( e.g. Inspect)

• Experience with LLM elicitation techniques ( e.g. fine-tuning, retrieval augmented generation, tool-use integration, agent scaffolding)

• Experience working on ML model development/deployment or working at/with leading AI companies

• Experience with c loud computing , in particular Azure and AWS , including government cl oud environments

• Familiarity with common LLM frameworks ( e.g. LangChain , LlamaIndex )

• Aptitude for project management

• Experience in government, intelligence community, other relevant decision-making offices, or policy analysis roles

Applicants who meet the required qualifications are strongly encouraged to apply; the preferred qualifications are not necessary.

Education Requirements

This role requires at least a Bachelor's degree in a relevant discipline. Higher levels of education ( e.g. PhD, JD, Masters) or experience are preferred. Relevant disciplines may include Artificial Intelligence, Machine Learning, Computer Science, Cybersecurity, Electrical Engineering, Physics, Mathematics, Engineering and Public Policy, Security Studies, or similar .

Note: This team also has an opportunity for a Research Scientist . Applicants that possess the required qualifications are welcome to apply to either or both positions.

Security Clearance
Ability to obtain and maintain a U.S. security clearance, including having US citizenship, is preferred but not required .

Location

We are actively hiring for this position in Washington, DC; San Francisco, CA; Boston, MA; Santa Monica, CA; and Pittsburgh, PA. San Francisco or especially DC are preferred . We offer a hybrid work arrangement, combining work from home and on-site options. Fully remote work will also be considered .

Term

This position is a 1-year term appointment with a possibility of renewal for up to 3 years total, alongside options for longer term employment.

Application

Applications must include:

• A detailed resume highlighting relevant academic and professional experience.

• A writing sample demonstrating analytical and communication skills. This sample may be a recent, previously written paper or report (e.g., journal article, masters thesis or paper written for coursework, prior employment, or internship). Applicants whose study and work experience (e.g., model development) has not involved producing written products that are shareable may submit a short, written summary (i.e., less than one page) of one or more recent products they have developed.

• A code sample. You can link to this from the bottom of your cover letter.

• A cover letter which contains only responses to each of the following prompts:

• 1) Summarize in .

• 2) Describe in For an infrastructure project: You may make guesses about our goals and existing infrastructure, and propose a way you might help improve that, noting how you would implement that, how many months of work may be required from you and/or colleagues, and wh y this might be useful. This is just an assessment step and does not mean you would definitely work on this if hired.

Salary Range: $47,100- $156,500

In RAND's title system, this posting is for the following titles and corresponding salary ranges:

Research Project Specialist I: $47,100 $68,100

Research Project Specialist II: $57,400 -$85,500

Research Project Specialist III: $75,700 $112,400

Research Project Specialist IV: $102,800 $156,500

This position is eligible for overtime .

RAND considers a variety of factors when formulating an offer, including the specific role responsibilities; a candidates work experience, education/training, skills, expertise ; and internal equity. In addition, RAND provides strong benefits including health insurance coverage, life and disability insurance, a savings plan, paid time-off, and more.

Equal Opportunity Employer