BRAINSTORMING - how to document things like depression for PEP natural medicine BB

We think there are a lot of things that can be done for psychological issues and mood disorders like depression that fall into the category of natural medicine . . . but the PEP format requires before and after evidence, usually a photo or video.

It has to be more than just someone saying what they did. So something like a questionnaire where the person with an ailment rates how severe their symptoms are, or something like that, does not meet Paul's standards.

We think we will just not be able to include these disorders in the PEP natural medicine stuff, and will have to stick to a limited number of physical disorders that you can document with imaging, blood tests, etc. But that would be a shame. So we are asking for help--can you think of any way to document improvement in someone with depression that meets the evidence standards of the PEP format?

One possible way would be an official diagnosis by a medical professional before vs after that confirms an improvement.  

Jennifer Richardson wrote:...
It has to be more than just someone saying what they did. So something like a questionnaire where the person with an ailment rates how severe their symptoms are, or something like that, does not meet Paul's standards.
...

This doesn't make sense to me because questionnaires are basically how doctors assess mood disorders. When getting a diagnosis, patients usually fill out a self screen questionnaire and show it to their doctor. If they go to see a psychiatrist or counselor, they are asked questions about how they feel. Treatment is started and the questions are repeated some time later. Asking the patient how they are feeling is the primary way that professionals determine if treatment is working or not.

The most widely used questionnaire: http://med.stanford.edu/fastlab/research/imapp/msrs/_jcr_content/main/accordion/accordion_content3/download_256324296/file.res/PHQ9_id_date_08.03.pdf
Second page in the link explains how scores inform diagnosis.

The questions:

1. Little interest or pleasure in doing things        
2. Feeling down, depressed, or hopeless        
3. Trouble falling or staying asleep, or sleeping too much        
4. Feeling tired or having little energy        
5. Poor appetite or overeating        
6. Feeling bad about yourself—or that you are a failure or have let yourself or your family down        
7. Trouble concentrating on things such as reading the newspaper or watching television        
8. Moving or speaking so slowly that other people could have noticed? Or the opposite—being so fidgety or restless that you have been moving around a lot more than usual        
9. Thoughts that you would be better off dead or of hurting yourself in some way

If you clicked on any problems above, how difficult have they made it for you to do your work, take care of things at home, or get along with other people?
Not difficult at all     Somewhat difficult        Very difficult      Extremely difficult

Mood disorders are tricky because most of the symptoms are happening in the patient's mind. We have to rely on their honest communication of their symptoms and believe their experience. There can be physical manifestations of symptoms, but not always. Some of the questions above allude to physical manifestations like length and quality of sleep, eating more less.

It seems like the best way to show changes in symptoms and quality of life would be a journal or calendar of some kind. Something like: before treatment suicidal ideation happened between 5-20 times per day, and after two months went down to once a week. I don't see how pictures or data from a journal is any less valid than pictures of blood test results.

Also if we require an official diagnosis, we will exclude people without access to such.

Tracking all of your symptoms for months could be a lot of work when it comes to certification, and a possibly insurmountable task for the depressed patient. Maybe each person could pick one part of their life that is most impacted by their depression that they want to improve and use that as a measure. The BB could be more roundabout than explicitly treating depression; it could be "improve quality of life" or something vague.

Mood disorders are also very individual as far as what treatments might work for each person. I think each person's submission will be very different and take a lot of time to certify. So maybe it's better to restrict the BB to trying a certain treatment and focus on documenting that the person tried the treatment in the BB. Whether it improves their life or not, if they tried the treatment, they get the BB.

I agree with Amy. There are many widely used and validated scales for screening for depression, assessing severity of depression, and response to treatment. These are not like the pop quizzes in Cosmo, but are genuine clinical tools used every day by mental health professionals. A validated scale for a mental health issue is just as much evidence as a photo is for say wound healing. All of the clinical trials on prescription antidepressants used some sort of validated scale to assess treatment outcomes.

The one I have most experience using (former maternal & child health nurse) is the Edinburgh scale: https://med.stanford.edu/content/dam/sm/ppc/documents/DBP/EDPS_text_added.pdf  It's very well validated for use with pregnant women and both women and men in the first year after becoming a parent, but less well validated for use with other groups.

The questionnaire Amy refers to is the PHQ-9. It's been around for many years and is recognized as a useful clinical tool for assessing severity of depression and response to treatment, is just as valid whether completed by the patient or done with a health professional, and would be appropriate for use in this context. Here's a good analysis validating the tool. The questionnaire and how to score it is in the appendix: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1495268/  It also has the advantage, unlike many of these tools, of now being copyright free. I also agree that whatever is decided as the assessment method needs to be something a depressed person can actually do. The simpler the better. For many severely depressed people, actually filling in a questionnaire or even seeeling treatment only becomes an option as the person begins to emerge from the worst of the episode.  

Maybe rather than "depression" which is a specific clinical diagnosis, it could be better labelled "persistent low mood"? I haven't read the rest of the PEP criteria for the natural medicine BBs, but it seems to me if someone needs a diagnosis from a qualified medical professional to be eligible for a natural medicine BB, it defeats the purpose to an extent!

I don't think anything other than the simplest medical conditions are a good fit for a specific BBs, for the reasons stated in the opening post.  We can't expect the moderators to diagnose whether a person actually suffered from a particular disease, and whether they recovered, and whether the recovery was due to the treatment.  

There is also the problem of setting up a BB which will only be available to people who happen to suffer from a particular disease, and the disease responds to their chosen treatment.

Maybe it would make sense to have a natural medicine BB that is more like the "oddball," something along the lines of documenting a longer-term natural medicine experience through journaling or other means, points given for explaining the health problem, why they chose a particular method of coping or treatment, and description of the positive (or negative) effects.

We aren't debating the validity of questionnaires for diagnosing mood disorders, it just doesn't fit the parameters of Paul's PEP stuff, so we can't include them. The PEP format requires video or photographic evidence; the whole point is to prove the skill of the person submitting the BB. A person could easily fill out a fake before and after questionnaire. They cannot easily fake an MRI of a tumor. So while questionnaires are accepted clinical practice, they aren't admissible for PEP, which is why we think we will have to exclude mood and psychiatric disorders, and why we started this thread.

Hmm, okay, that makes it tricky. There's zero way to have any photographic evidence for persistent low mood or depression.

Using your example of an MRI image of a tumor, it seems to me that if someone is going to go to the lengths of faking questionnaire evidence, they can just as easily and quickly fake an MRI image. Simple to switch out the patient name, if the individual has any skills with Photoshop or other image editing apps. MRI video would be harder to fake but are huge files that could be challenging to send electronically for verification. Unless you want actual photographic evidence - here I am in the MRI machine, which wouldn't be possible anyway as patients can't take cellphones into the imaging room - it's hard to imagine how anything but visible external health conditions can be conclusively verified. It's a little more work to fake journal entries showing symptoms of depression, but make the task too complicated and the depressed person won't be able to do it.

Journaling, as Amy and Mk already mentioned, seems to me the most useful way to document otherwise invisible health issues. I like MK's description of what a journal could include. Symptoms, rationale for choosing a specific treatment, photos showing the person applying for the BB preparing the remedy or in some other way verifying they did the treatment, results of the treatment. Still fake-able, and opens a whole other can of worms of who decides whether the person being treated actually has/had the condition, and whether the treatment chosen was the most appropriate, plus time consuming for the person validating the BB to assess.

I guess I should butt out until I've seen what is proposed to verify the other natural medicine BBs, apart from the very basic ones, as I don't have any useful suggestions to add. This whole issue of verifying treatment for complex health conditions has the potential to be even more complex than treating the health condition!

Yeah, it's really tricky. We're settling on kind of a hybrid.

So for conditions, no matter how serious, in which the only real metric is basically "sick person says they feel better" (whether it's a sore throat or something like depression or lupus), we are assigning only half a point. You have to provide a description of the issue they're experiencing, what you suggested, and how it worked, and a before and after picture of you and the sick person giving a thumbs up or something. It doesn't prove that what you did worked, but you have to at least convince someone to take a picture with you and do some documentation, which wouldn't be worth faking for half a point.

And then relatively simple but concretely documented things (iron deficiency anemia, high blood pressure) get a full point.

And curing someone's cancer gets mega points (tens or hundreds of points, depending).

Maybe for something like depression, if you do major documentation like keeping a daily log of things you tried (exercise, low glycemic load diet, supplements, whatever) and your various symptoms on a scale for several weeks, we might give you 3 points. Which doesn't reflect the seriousness or effort that went into it, but rather the lack of concrete metric.

For stuff like faking imaging tests, some of the countermeasures might be things like it has to be a picture of the patient holding their x-ray or something (making it harder to just pull something from the internet), and if you ever get caught cheating, you are banned from the whole program forever and lose all your work (on any of the various PEP aspects). Not foolproof, but harder.

It's all really squishy and complicated, and Paul has thought multiple times about scrapping natural medicine, but we are trying to find a way to make it work because we think it's really valuable.

It's definitely an essential skill. I guess the question is how far to take it with specific health conditions. What you're proposing sounds good and should be workable.
All this PEP documentation - it's a huge and valuable piece of work you're doing!

Quotes I agree with:

“I don't think anything other than the simplest medical conditions are a good fit for a specific BBs, for the reasons stated in the opening post.  We can't expect the moderators to diagnose whether a person actually suffered from a particular disease, and whether they recovered, and whether the recovery was due to the treatment.” (Mk Neal)

“... maybe it's better to restrict the BB to trying a certain treatment and focus on documenting that the person tried the treatment in the BB. Whether it improves their life or not, if they tried the treatment, they get the BB.” (Amy Arnett)  

And that's why I think too this will be a good solution:

“So for conditions, no matter how serious, in which the only real metric is basically "sick person says they feel better" (whether it's a sore throat or something like depression or lupus), we are assigning only half a point. You have to provide a description of the issue they're experiencing, what you suggested, and how it worked, and a before and after picture of you and the sick person giving a thumbs up or something. It doesn't prove that what you did worked, but you have to at least convince someone to take a picture with you and do some documentation ...” (Jennifer Richardson)

My instinct is to steer away from proving a treatment worked, and focus on proving a widely accepted, approved treatment was administered, or perhaps grown/preserved. I think this would be on par with the effort required for many other lower level BB's.

Jordan Holland wrote:My instinct is to steer away from proving a treatment worked, and focus on proving a widely accepted, approved treatment was administered, or perhaps grown/preserved. I think this would be on par with the effort required for many other lower level BB's.

I agree with this. I think for a lot of treatments, whether the treatment "worked" or not is subjective and not proveable through any readily available objective means. Most natural medicine remedies are not one-shot cures, but methods of alleviating symptoms to a lesser or greater extent, or hastening the body's natural healing. Just as an example, witch hazel is widely accepted remedy for inflammation, redness, irritation from skin problems, hemerrhoids, bug bites. It definitely provides relief.  However, it does not make any of these instantly problems disappear. The person just feels better after each application, and the problem might resolve itself sooner than without treatment, though no way of proving this in any given instance (that's what double-blind studies are for).

Just pitching in because this thread suddenly got revived.

Mk Neal wrote:My instinct is to steer away from proving a treatment worked, and focus on proving a widely accepted, approved treatment was administered, or perhaps grown/preserved. I think this would be on par with the effort required for many other lower level BB's.

I also agree with this. Seeking proof of something being the direct cause of recovery is something which even modern medicine struggles with. Broken bones will eventually heal, so who's to say whether the comfrey poultice actually helped?

Preparation and administration being separate BB tasks certainly seems like a good move to me.

Jennifer Kobernik wrote:We aren't debating the validity of questionnaires for diagnosing mood disorders, it just doesn't fit the parameters of Paul's PEP stuff, so we can't include them. The PEP format requires video or photographic evidence; the whole point is to prove the skill of the person submitting the BB. A person could easily fill out a fake before and after questionnaire. They cannot easily fake an MRI of a tumor. So while questionnaires are accepted clinical practice, they aren't admissible for PEP, which is why we think we will have to exclude mood and psychiatric disorders, and why we started this thread.

So it looks like this brainstorming thread ended in a no go way back in 2020.

I’ve thought before that if there is a way to do this, a testimonial quote is probably the way—but in no way is that photographic evidence.  

I also find many aspects of certain parts of PEP to be unworkable in consideration my standards of internet privacy—pictures of faces, posting medical information on the web (!), etc. It makes me wonder if there could be a way of submitting privately such that only real otises could see.