To give an update: the Support team finally nailed the core issue. Here’s their response:
The download process can be slow, especially if you are downloading a large number of small files (less than 1 MB each). These files need to be downloaded in the background one by one to ensure that each download is successful and the file is properly decrypted. As a result, the download speed cannot exceed a certain threshold since the client only downloads/uploads up to 3-4 files simultaneously.
For example, if the 4 files being downloaded are 1 KB each, the download speed would be limited due to the small file size and the sequential nature of the process. Our developers are actively working on improvements to enhance this aspect of Proton Drive, aiming for faster file downloads. However, we cannot provide an exact date for when these improvements will be implemented.
Idk how they found out, maybe by looking at the meta data (?), but indeed the 200mb files I was downloading were comprised of a lot of individual files of <1mb each. I tried downloading just a single piece of video of 119mb and it took a couple of secs (both on their app or the web version).
No further action from my end and hope they get to improve this sooner than later.