Dpt Computational Biology Methods Primer “Foundation models of RNAseq data and Biological Networks” – Jeremie Kalfon

Dpt Computational Biology | Methods Primer “Foundation models of RNAseq data and Biological Networks” – Jeremie Kalfon

2025-01-20

• Modeling ScRNAseq
  • Gene network, via machine learning
  • Transformer » Neural Network (NN) (e.g., AE, ScVI)
  • Input matrix, col - cells, rows - genes.
  • Different type of methods:
    • graph based
    • decomposition
    • MF
• NN
  • ReLU: Non linearity
  • Stochastic gradient descent
• Auto encoder
  • Dimension reduction, works like PCA?
• ScVI
  • Loss function: $L_{VAE} = \prod{SE} + \beta KL(p\vert\vert q)$, $p$ is prior, $q$ is a prediction.
• Transformer
  • linking input matrix to output matrix
  • Inside: SA, MLP
  • skip connection
  • attention matrix
  • good for scaling
  • SAs (self attention): $softmax(XQK^TX^T/\sqrt{d_k})XV$
• GPT
  • There is encoder and decoder but it is not the same as those mentioned above.
  • Maybe it can help to find link between genes, forming the gene network.
  • Order of genes can be identified.
• DNA transformer