Scott Gray
Member Of Technical Staff at OpenAI
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolScott Lessans is a Member of Technical Staff at OpenAI with 11 years of experience building high-performance ML infrastructure, robotics software, and startups from San Francisco. He combines product leadership as a former co-founder and CTO with deep systems expertise from roles at Covariant and cr8dl.ai. An active open-source contributor to projects like Intel Nervana's neon and OpenAI's blocksparse, he has implemented and optimized GPU kernels, convolution/pooling engines, and fused operations to improve performance across hardware (including sm_50 support and asymmetric query/key handling for sparse transformers). He holds a Computer Science Engineering degree from the University of Michigan and brings entrepreneurial grit alongside low-level ML systems optimization skills.
11 years of coding experience
Physics, Computer Science, Physics, Computer Science at University of Illinois Urbana-Champaign
Github Skills (21)
transformers10
sparse-array10
convolutional-neural-networks10
gpgpu10
batch-normalization10
python10
sparse-matrix10
gpu-programming10
machine-learning10
sparse-file10
kernel10
deep-learning10
tensorflow10
gpu10
performance-optimization10
Programming languages (5)
Github contributions (5)
openai/blocksparse
Jan 2018 - Jul 2019
Efficient GPU kernels for block-sparse matrix multiplication and convolution
Role in this project:
ML Engineer
ML EngineerContributions:21 commits, 3 PRs, 21 pushes in 1 year 6 months
Contributions summary:Scott primarily contributed to the development and optimization of GPU kernels within the blocksparse library, which focuses on efficient sparse matrix operations. Their work includes implementing support for specific hardware architectures like sm_50, improving performance through code changes, and adding features such as fused softmax_cross_entropy operations. Furthermore, they integrated support for asymetric query/key dimensions, enhancing the library's applicability to diverse machine learning models, particularly in the context of sparse transformers.
cudamatrix-multiplicationsparse-matrixgpusparse
NervanaSystems/neon
Sep 2015 - Jul 2016
Intel® Nervana™ reference deep learning framework committed to best performance on all hardware
Role in this project:Backend Developer
Backend DeveloperContributions:36 commits, 35 comments in 9 months
Contributions summary:Scott primarily contributed to the development and optimization of the Intel Nervana deep learning framework, specifically focusing on improving the performance of convolutional neural networks. Their commits include implementing new engines for convolution and pooling, enhancing batch normalization, and creating a new build system for kernels. These improvements were benchmarked and tested to ensure optimal performance on various hardware platforms.
deep-learningbest-performanceintelmachine-learningperformance
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.