SophontAI's Tanishq Mathew Abraham argues medical AI must scale on raw clinical data rather than lossy human heuristics

@matt_is_nice These are ideas that have been tested for over 15 years and didn’t really pan out (radiomics). You might find some very small signals here or there

@alwaysinscope and this one hasn't even taken vc money! just a bootstrapped company taking a big swing at improving medicine!

If you can measure it, you can automate it.

@mattbeane @CanesDavid Yea this is a very good point. That’s why I like the spa angle and targeting the longevity/wearable crowd. Tell them honestly, “we don’t know what this data means yet, so we’re not going to show you. But we’re capturing data that will be useful later!”

Sorry I’m not following you here. The labeled data in this case is just other health outcomes data generated via standard care.

This is what I would like to see Midjourney do:

1) Collect scans via spas or whatever scaled delivery 2) Do nothing with that data for a long time 3) When critical mass is reached, use that raw data to train a self-supervised embedding model 4) Run a series of observational and prospective studies under IRB to determine what those embedding are capable of detecting/predicting

@matt_is_nice You know what else resonated with technologists especially but not with the medical establishment?

This is close but actually the way we trained our endoscopy model was to first just capture an incredibly large volume of unlabeled endoscopy data. Millions of procedures.

We then used self-supervised learning to train an encoder model for endoscopy. Turns out this works incredibly well without labels. The model learns its own powerful representation for endoscopy.

We can then apply that encoder model to smaller labeled datasets and get amazing results.

This is what the Midjourney Scanner can/should do IMO

@matt_is_nice agreed. why not let vc's mature a medical diagnostic, its a win win worst case we have another diagnostic, best case its revolutionary, certainly no revolutionary upside potential with juicero

Yes! The embeddings from the self-supervised model eventually are used with some sort of labeled data…biopsy results, histopath scores, patient reported outcomes, surgical outcomes etc etc. that’s this part:

“We can then apply that encoder model to smaller labeled datasets and get amazing results.”

But the key is that the self-supervised embeddings enable better predictive performance with smaller labeled datasets. This is state of the art practice in basically all computer vision, not just medicine. Where medicine has struggled is in building the massive unlabeled datasets needed to train the self-supervised models.

That’s why the Midjourney Scanner concept is exciting!

@matt_is_nice so you're ok with using some humans as guinea pigs to get the labeled data then...

@matt_is_nice I worked with a startup to use ultrasound + ai for cheap early detection of lung cancer. Huge pressure against it.

@mattbeane I'm less concerned about false positives since we can sample more frequently, and AI will only improve this over time. We should back approaches that challenge the status quo from first principles!

@matt_is_nice Those unlabeled models were then correlated with biopsy results. I'm sure of it If not, show an actual reference to the methodology

Good post.

Unfortunately, HIPAA breaks the link between scan results and future health outcomes, by making it, perhaps not technically impossible, but legally infeasible to attach patient identifiers to outcome data that passes between firms.

The practical implication is that AI can only be trained to replicate human judgement, since the human and the AI can look at the same scan without HIPAA entanglement.

It's another tedious example of the Law of Unintended Consequences.

@matt_is_nice "The actual goal should be to capture as much raw signal of a person's clinical state as possible."

Wrong. This is your *business goal*, because you are trying to sell AI.

That does not mean it should be everyone's goal. That would be retarded.

@quardart_seisey It’s even worse! Before we came along they wouldn’t even capture the video! Just disappeared never to be seen again. We’re turning the tide on that though!

@matt_is_nice @CanesDavid Ohhhhhhh interesting. If you could pull it off, ideal.

But that would mean withholding that data from patients? I mean, humans would want to know their own data, would start choosing to talk to docs, ask for further tests...

So that seems really hard to do.

@_fiziks What does low quality mean? How do we know if it contains little useful information?

The approach I outlined is the correct way to answer those questions, I think.