Responsibilities:
Scope, coordinate, and procure data collection and labeling projects and identify and capitalize on opportunities to make them more efficient.
Monitor ongoing projects to make sure that resources are directed appropriately, adjusting scope and instructions with new information.
Develop and maintain strong relationships with external vendors and contractors, ensuring that our work together is high-value and low-friction.
Execute on human data projects, doing activities such as writing labeling instructions, reviewing data, and making sure we have the right mix of quality, diversity, and volume of data, as well as figuring out how to improve the tooling or instructions to make things more efficient.
Report out on projects, including dashboards and data analysis.
Partner with Engineering to maintain and improve our human feedback interface.
Develop and support the operations for an in-house team of human data workers.
You might be a good fit for this role if you:
Are extremely detail-oriented and enjoy poring over information and data line-by-line.
Have exceptional project and relationship management skills, with both internal and external parties.
Have a strong understanding of how LLMs work and of prompt engineering.
Have experience with data collection, labeling, and analysis.
Have experience using tools like SQL, Python, Tableau, etc.
It is particularly valuable if you:
Have experience with human data collection and labeling specific to large language models.
Have managed a team of external contractors and/or vendors before.
Have conceived of and launched new technical programs.
Have worked with researchers and engineers.
The expected salary range for this position is:
Annual Salary:
$290,000 – $365,000 USD
Education requirements: We require at least a Bachelor’s degree in a related field or equivalent experience.
