AVERAGER STUDENT: DISTILLATION FROM UNDISTILLABLE TEACHER

Reyhan Kevser Keser; Behçet Ugur Töreyin

AVERAGER STUDENT: DISTILLATION FROM UNDISTILLABLE TEACHER

Informatics Institute

Research output: Contribution to conference › Paper › peer-review

1 Citation (Scopus)

Abstract

Today, some companies release their black-box model as a service for users, where users can see the model’s output corresponding to their input. However, these models can be stolen via knowledge distillation by malicious users. Recently, undistillable teacher (Ma et al., 2021) is introduced in order to prevent the knowledge leakage. In this study, with the aim of contributing to solutions for model intellectual property (IP) protection, we propose a novel method which improves the distillation from an undistillable teacher whose goal is make the distillation difficult for students, with the purpose of model protection. The codes are released at https://github.com/rkevser/AveragerStudent.

Original language	English
Publication status	Published - 2023
Event	1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023 - Kigali, Rwanda Duration: 5 May 2023 → 5 May 2023

Conference

Conference	1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023
Country/Territory	Rwanda
City	Kigali
Period	5/05/23 → 5/05/23

Bibliographical note

Publisher Copyright:
© 2023 1st Tiny Papers Track at ICLR 2023 - Tiny Papers @ ICLR 2023. All rights reserved.

Cite this

@conference{510a42fb522847e9b98768e63ec3d83d,

title = "AVERAGER STUDENT: DISTILLATION FROM UNDISTILLABLE TEACHER",

abstract = "Today, some companies release their black-box model as a service for users, where users can see the model{\textquoteright}s output corresponding to their input. However, these models can be stolen via knowledge distillation by malicious users. Recently, undistillable teacher (Ma et al., 2021) is introduced in order to prevent the knowledge leakage. In this study, with the aim of contributing to solutions for model intellectual property (IP) protection, we propose a novel method which improves the distillation from an undistillable teacher whose goal is make the distillation difficult for students, with the purpose of model protection. The codes are released at https://github.com/rkevser/AveragerStudent.",

author = "Keser, \{Reyhan Kevser\} and T{\"o}reyin, \{Beh{\c c}et Ugur\}",

note = "Publisher Copyright: {\textcopyright} 2023 1st Tiny Papers Track at ICLR 2023 - Tiny Papers @ ICLR 2023. All rights reserved.; 1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023 ; Conference date: 05-05-2023 Through 05-05-2023",

year = "2023",

language = "English",

}

TY - CONF

T1 - AVERAGER STUDENT

T2 - 1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023

AU - Keser, Reyhan Kevser

AU - Töreyin, Behçet Ugur

PY - 2023

Y1 - 2023

N2 - Today, some companies release their black-box model as a service for users, where users can see the model’s output corresponding to their input. However, these models can be stolen via knowledge distillation by malicious users. Recently, undistillable teacher (Ma et al., 2021) is introduced in order to prevent the knowledge leakage. In this study, with the aim of contributing to solutions for model intellectual property (IP) protection, we propose a novel method which improves the distillation from an undistillable teacher whose goal is make the distillation difficult for students, with the purpose of model protection. The codes are released at https://github.com/rkevser/AveragerStudent.

AB - Today, some companies release their black-box model as a service for users, where users can see the model’s output corresponding to their input. However, these models can be stolen via knowledge distillation by malicious users. Recently, undistillable teacher (Ma et al., 2021) is introduced in order to prevent the knowledge leakage. In this study, with the aim of contributing to solutions for model intellectual property (IP) protection, we propose a novel method which improves the distillation from an undistillable teacher whose goal is make the distillation difficult for students, with the purpose of model protection. The codes are released at https://github.com/rkevser/AveragerStudent.

UR - https://www.scopus.com/pages/publications/105026756305

M3 - Paper

AN - SCOPUS:105026756305

Y2 - 5 May 2023 through 5 May 2023

ER -

AVERAGER STUDENT: DISTILLATION FROM UNDISTILLABLE TEACHER

Abstract

Conference

Bibliographical note

Other files and links

Fingerprint

Cite this