A repository containing my submissions for the evaluation test for prospective GSoC applicants for the CMS and End-to-End Deep Learning Project.
I am applying for the project Vision Transformers for End-to-End Particle Reconstruction for the CMS Experiment.
Note- Use NBViewer to view the jupyter notebooks, as Jupyter notebook cell outputs are not rendered on GitHub. Or clone the repository to local for viewing Jupyter notebook cell outputs. Or view the PDF files, corresponding to task notebooks.
NBViewer Link: Task 1 Tensorflow Notebook
NBViewer Link: Task 1 Pytorch Notebook
I have used different models for Pytorch and Tensorflow Implementation.
Feeded hit energy and time channels to different Input layers.
| Train AUC Score | Validation AUC Score | Best Validation AUC Score |
|---|---|---|
| 0.8008 | 0.6805 | 0.7895 |
In Task 1 Tensorflow implementation, I have used keras.metrics.AUC as the evaluation metric.
The validation AUC score and loss is fluctuating (I have tried with different learning rates, optimizers, models and layers' parameters, but the issue remains the same).
Feeded both hit energy and time channels to Conv2D input layer.
| Train AUC Score | Validation AUC Score |
|---|---|
| 0.74 | 0.71 |
In Task 1 Pytorch implementation, I have used sklearn.metrics.roc_auc_score as the evaluation metric.
NBViewer Link: Task 2 Notebook
I have used the run0 file dataset for this task. I have used Tensorflow for model implementation.
Feeded X_jets, pt, m0 to different Input layers.
| Train AUC Score | Validation AUC Score |
|---|---|
| 0.785 | 0.771 |
In Task 2 implementation, the challenge was that numpy was detecting X_jets as a numpy object, instead of ndarray. numpy's inbuilt functions were taking unbounded time to process X_jets into ndarray object. So, I used four nested loops to copy the data into ndarray object. I have used 5x5 kernel size and pool size for Conv2D and MaxPooling2D, owing to the sparse data.
NBViewer Link: Task 3 Notebook
I have used Vision Transformer based Model, with Dropout rates as 0. As the data is sparse, we cannot afford Dropout layers in the Transformer model. Used softmax as activation for last Dense layer for Electron and Photon Binary Classification. Model Implementation is in Tensorflow.
| Train AUC Score | Validation AUC Score | Test AUC Score |
|---|---|---|
| 0.806 | 0.782 | 0.777 |
In Task 1 implementation using CNN, the validation loss and AUC score was fluctuating for the Tensorflow implementation model. In Task 3 implementation using Vision Transformer, the validation loss decreased in proportion to the training loss upto validation loss of 0.56, corresponding to validation AUC score of 0.78. I have used categorical_crossentropy loss function.