Chenghao Li

Chenghao Li

M.S Student in Machine learning and data science

Private Inference(PI)-friendly Visual Transformer Structure

(01/2023-05/2023)

  • Built SAL-ViT to boost PI efficiency (maintain accuracy & reduce latency) on ViTs with Pytorch.
  • Developed learnable 2Quad (L2Q) as the approximation of Softmax, which introduces learnable scaling and shifting parameters to the prior 2Quad, and trained on Cifar and ImageNet with knowledge distillation.
  • SAL-ViT can averagely achieve 1.60×, 1.56×, 1.12× lower PI latency with 1.79%, 1.41%, and 2.90% higher accuracy compared to the existing alternatives, on CIFAR-10, CIFAR-100, and Tiny-ImageNet, respectively.
  • Authored(has been accepted) a paper in 2023 International Conference on Computer Vision(EI).
  • Authored(has been accepted) a paper in 2023 International Conference on Computer-Aided Design(EI).