Sci-hub is your friend. The domain changes periodically, which is why I'm not linking it, but you can just give it the DOI of the article and voila, there's the PDF.
figure 3 A says: "During the development of depression, susceptible neuronal networks become gradually
hyper- (red) or hypoactive (blue) under predisposing environmental conditions, and they retain that state at the expense of optimal network
functionality. This may manifest as, for example, uncontrollable self-focused rumination, depressive thoughts, anhedonia, and cognitive dysfunctions.
Rapid-acting antidepressants (i.e., ketamine represented in the figure) have excitatory effects that are reflected in the activity of local circuits, leading
to increases in cortical synaptic strength and the re-emergence of global functional connectivity patterns. This breaks the recursive cycle of rumination
and allows for the immediate relief of depressive symptoms."
Not a typo, but the divide by 100 example of renormalization is wrong. Normalization implies that the vector is being made normal, for example having a "normal" magnitude. A better example would be dividing all the numbers (0.9+0.5+0.3), so that they add up to 100.
I'm not sure about how AI-ers use the idea of renormalization (a quick search turned up a paper titled "Neural Network Renormalization Group", which putatively describes neural networks using a renormalization-type idea from physics, and a number of other completely unenlightening papers/writings), but presumably I think it would mean just dividing stuff by an appropriate constant to fix its scale. Also sqrt(0.9^2 + 0.5^2 + 0.3)^2 = 1.073 ~ 1, so even if we are uncharitably assuming the exact mathematical definition of renormalization Scott Alexander's example is not too far off the mark.
Not totally sure what Scott actually meant by this, but the two almost ubiquitous practices in statistical modeling in general (not just deep learning or layered models) are 1) clamping data values to a small continuous range like [-1, 1] and scaling everything based on distribution parameters of the known inputs, and 2) penalizing complex models via something like L1/L2 shrinkage (the minimum description length principle from a Bayesian perspective). The former I would call "normalization" (typically you're assuming a normal distribution, which means you can just do x' = (max(xs) - x) / (max(xs) - min(xs)), maybe after removing some obvious outliers, and you've just clamped) and the latter I would call "regularization."
I don't know much about physics, but don't *think* this is the same thing as renormalization in that field, except in the sense that both involve making assumptions about data and manipulating numbers based on that. This seems like quite a bit more of an exact science, so to speak, in physics, just because the data generating processes are much better understood.
Although, for what it's worth, when L1 regularization first came to linear regression analysis in the 80s (L2 had already long been in use and was known as "ridge regression"), the idea also came from physics. Normalization in the sense I'm talking about is different. There is no theoretical basis from probability guiding why you'd do it. You're doing it because of numerical stability of the training process, to avoid having your gradient estimates blow up because of really big input values. Regularization, on the other hand, involves making theory assumptions about the nature of the maximum a posteriori model.
Is it silly to notice the superficial similarity between EMDR and REM (both involve moving the eyes) and effects on psychiatric conditions? Might the evidence for the mechanism of action for one be relevant to the other?
Eye movements are controlled by the frontal eye field and are typically the only part of the body unaffected by atonia during REM sleep, thus REM occurs as we dream. There are two hypotheses of the cause of EMDR efficacy in treating trauma. The first is that eye movements or a different bilateral stimuli such as tapping or vibrations (yes, EMDR does not always involve eye movement!) are in some way responsible for “activating both hemispheres” or some equally hand waves and implausible mechanism. It might be right but it doesn’t make sense to me. The second hypothesis is that the eye movements or bilateral stimuli are purely placebo and that what really works is imaginal exposure. Having successfully treated PTSD with trauma focussed CBT alone, I favour the second hypothesis. I’d therefore conclude that the eye movement is a coincidence but would happily change my mind if a plausible eye movement in EMDR theory was put forward
There's also the theory that following the alternating sides stimuli in EMDR continuously triggers an alarm reaction.
The alarm reaction is hard wired and partially blocks the remembered trauma, so that it is not able to overwhelm (and thus not desensitize, or even re-traumatize) the patient.
Anecdotally (both from applying or being treated with EMDR), jagged irregular zigzagging motions of the therapist's hand waves require a stronger focus in the patient, are triggering a stronger alarm reaction (as felt in one's alertness), and thus have a stronger shielding against flooding effect.
Also the classic hypnotist's pendulum - "you are getting very sleepy...". There's a lot of stuff like this and I don't have a good understanding of any of it.
In a recent post you hypothesized that "paying special attention to the sensory input channel" was important in EMDR. Is there a just-so story we can apply to REM that's consistent?
"The ability to look upward on signal while closing the eyelids (Eye Roll Sign) correlates highly (73.9%) with hypnotic trance capacity as measured by the Hypnotic Induction Profile in 2000 consecutive psychotherapy cases. In practical clinical terms this implies that in about five seconds the Eye Roll (ER) sign predicts hypnotizability in three out of four cases. Further, the higher the roll (0–4 scale), the higher is the trance capacity. This offers a quick, subtle, clinical way to ascertain whether or not hypnosis can have a probable role as an adjunct in the various psychotherapeutic strategies."
Have caution. In looking up just what the "Eye Roll Sign" was to better understand that first sentence, a top Google result was a 1982 paper whose abstract claims that the purported correlation is far weaker than advertised, to the point of being clinically useless: https://jamanetwork.com/journals/jamapsychiatry/article-abstract/492859
I just tried it and I found that I can kind-of do it. Is this 🦋 depressability*?
*For the confused: (putatively) passing Spiegel Eye Roll Test -> being able to be hypnotized -> being able to do EMDR -> lots of REM sleep -> depression.
It seems like I can do this, but I tried very hard to get hypnotized back when I was an impressionable teenager and got really into Edward Cayce and was convinced human clairvoyance was both real and something I could tap into if I could just manage to achieve a higher plain. It never worked. I was never able to be hypnotized.
DISCLAIMER: please see Majromax's response. I agree with it. And should have included in my initial comment that I wasn't endorsing the information as accurate, but just meant to provide another data point for eye movement being a thing that the hypnotism community cares about, and a slightly more concrete example than "yeah they do the thing with a pendulum".
As a data point, I am sometimes depressed, but always feel least shitty in the morning, where I have the most energy and mental clarity, and feel most shitty at night. This seems to be one of those axes along which there's your pretty typical spread of variation; if depressives seem to disproportionately be more night owl than morning lark, that *could* be due to the effects speculated here, or it could just be an "everything correlates with everything else" thing.
On this blog I've heard a bunch of times about how sleep deprivation can help (temporarily) with depression, but out in the normieverse I only ever hear about how sleep hygeine is good for mental health. Granted that there are other reasons why "permanently sleep deprive yourself" (which depression might do anyway lololol) isn't a good idea, is this just the broader culture *assuming* that a generally healthy thing (consistent, full sleep) is good in every specific, or is it still best, including for depression? (It's possible this was answered somewhere in the technical details that I lack the brains and background to follow.)
To me, 'sleep hygiene' has always meant 'stable circadian rhythm', which I suspect is probably more effective than mere sleep deprivation in the long-term treatment of depression.
I will second this experience -- when I went through a several-month period of severe depression several years ago, I always felt relatively fine in the morning (just anticipatory), horrible by midday, and deathly by evening. My desire for sleep (general sleepiness throughout the day also surged), and I started sleeping far more: normally I sleep 5-7 hours if I do not set an alarm, but the amount I sleep surged to 11-15 hours without an alarm, and budgeting ~8-9 hours per weeknight (which was the most I could realistically give myself) seemed terribly insufficient.
After about four months of this it pretty much went away after a psychedelic trip.
Most of the time I have a similar pattern as you, feel crappy in the morning, feel crappy at night, feel less crappy in-between. This is actually the reason why I don't get enough sleep, even if my mid-day self says "I need to go to bed at 10 PM", my night-time self sometimes says "Tomorrow is hopeless so let's stay up till 3 AM to keep away the pain."
But I did have some interesting experiments in college with sleep deprivation. Part of this may have been that I was more manic in my college years (my illness is probably some sort of type 2 bipolar, although the manic side has become less prominent over the years, and this is coupled with some degree of an anxiety disorder), but I found if I reached a certain level of tireness I was too tired to think.
This was amazing for me, suddenly I could just tell myself do this and I would do it without any negative back-talk or anxiety wrestling. This happened reliably enough that I tried to maintain that level of tireness for one or two weeks. Of course the downsides showed me that this was unsustainable. My body felt like it was falling apart and because the whole point was I was too tired to think, I had trouble thinking deeply about things which began to affect my ability to learn new concepts. Eventually I had to catch up on my sleep.
I don't think what I'm talking about necessarily has anything to do with what Scott was writing about, but that experience has made me interested in the relationship between mental illness and sleep.
As in my other comment, I'm wondering whether the significant fraction of people with sleep issues who have depressive symptoms might be contributing to this axis. I.e. a lot of people with depression may feel bad in the morning because a lot of people with depression (>20%) actually have a sleep issue like sleep apnea that causes both feeling-bad-in-the-morning and depression.
This seems like the kind of thing people would have thought of, but then again, it seems like depression patients should be tested for sleep issues (if not right away, then at least after failing a typical antidepressant + therapy attempt), and that's not widespread.
I used to have depression-like-but-clearly-not-typical-depression issues that I eventually discovered grew out of a combination of sleep apnea and dust mite allergy. Up to a threshold of exhaustion, I'd definitely feel better the longer it had been since I'd slept, and there was probably an element of feeling better the longer I'd been away from my mattress/pillow. Having controlled both of those problems, I no longer have any depressive-like symptoms.
This has been my experience with people who suffer from depression as well - people who are depressed often feel worst when they are tired. My thought was always that tiredness led to them being less able to push back against negative mental patterns, and that being tired made standard mental patterns more common/prevalent as the body tries to shortcut as much as possible on thinking.
Honestly, "depression" probably isn't actually one disease at all, but a symptom of a number of different diseases, which is why people with "depression" respond so differently to different treatments - some of them have one condition, some another. Some people have "anxiety depression", other people have "bipolar depression", some people have "sleep depression", ect.
The fact that a lot of people with depression also have problems sleeping makes me very suspicious of the idea that sleep deprivation helps depression, as people with disordered sleep - and who thus end up tired more often - end up *more* depressed, not less, on average.
My experience with depression and sleep deficit has been it's worse at least in part due to...not pain, exactly, but the discomfort that comes with tiredness. It's hard to feel good, or even okay, when you've got a headache and your eyes hurt and you've got an ache in your muscles because you haven't gotten enough rest and your chest feels weird because you've drunk too much caffeine.
I feel crappy immediately after waking up (specifically before I get up and start having inputs, leaving me with just brooding), crappy at night (despite the inputs), better in-between.
I recognize this far more than the example in the article. I must ask, how fatigued were you? How anxious? Both of these can ramp up during the day by fairly obvious means, and lead to depression through unsophisticated channels.
If I understand this right, maybe the better sleep hygiene thing recommended for depression needs to be amended with restricting the amount of sleep you get, if you tend to oversleep like I do.
Would it make sense to "jumpstart" synapse density first thing every morning by reading/learning? Perhaps that habit would help keep depression at bay.
As someone who often feels depressed in the morning, I have noticed that if I do something especially positive in the morning, it immensely helps me the rest of the day. The difficult thing is that when I wake up I feel like nothing is going to work, nothing is going to help, I don't even deserve to be happy. So even if I have a plan of, "okay, I'm going to take a 30 minute walk to start out the day", when I wake up in the morning I throw the plan away and then I get angry at myself for throwing the plan away which leaves me sometimes more depressed than if I did not have a plan in the first place.
But hope springs eternal, I do want to get into a pattern of doing something positive when I wake up, I still think it's a good idea and I still plan on trying... but it's hard.
You wake up early and do several “savers”. “Savers” are different tasks that you perform like visualization, meditation/prayer, reading and journaling. I think there are more, but that’s the gist.
Have you tried a "sunrise" alarm clock? I found that helped significantly with morning malaise for me--the gradual wake-up takes a lot of the sting out of it. I made mine out of "smart" lightbulbs, just set a program for them to start gradually getting brighter and bluer over an hour every morning.
I wanted to explore this same idea. I notice I'm in a much better mood on a workday vs a weekend or holiday. I've noticed this for years and wondered if I just felt more distracted and preoccupied at work and didn't pause to think about mood during the day. This gives a different explanation. Further, I've noticed I like to keep my laptop by my bedside and start my day working on spreadsheet activities. I've recently come to think of it digital gardening. And again, this article gives a new explanation as to why this feels satisfying to do.
I didn't look too long but I'm wondering if to what extent they have tried extended sleep deprivation as a treatment. The link you provided seems focused on these 36 wake periods in the short term, not the long term. As in, rather than just see how symptoms are reduced over the course of one or five cycles of 36 hour wake periods or whatever number they've chosen, how are they reduced if this sleep deprivation is done as a consistent lifestyle? Obviously this would pose a problem with living your normal life (family or work obligations, etc.), but if you have a severe case of depression those may not be functioning well anyway and you might as well give this a try if a researcher is offering it.
I know the depression seems to come back as soon as they fall asleep again, but perhaps with enough persistence this effect could be reduced through some sort of positive feedback loop, knowing that at the end of the day you'll feel remarkably better and you only spend maybe half your time feeling like crap instead of all the time.
There are a lot of experiments with trying to find better ways to make this last long-term. The most effective non-pharmaceutical one seems to be a week of on-and-off sleep deprivation culminating in your circadian rhythm permanently shifting to a different bedtime. I'll probably have a Lorien Psych post up with the specifics at some point.
I expect in the process of coming up with that they tried something like what you suggest, but I'm not sure.
I always ask depressed patients about sleep. One strange thing in DSM and ICD criteria for major depressive disorder is the hypersomnia or hyposomnia. Too much sleep or not enough? Aren’t those completely different symptoms?
I like Tononi’s synaptic normalisation hypothesis and there seems to be good evidence for it. There’s also evidence that a lot of metabolic waste products and extra cellular debris are cleared from the brain via the ventricular system during sleep, so it’s possible there could be more than one purpose for the change in activity.
I seem to recall that the build up of substance P during sleep was proposed as being connected to the depressogenic nature of sleep. Conversely, the build up of adenosine during the day, to make us sleepy, was proposed as an antidepressive effect (I can’t remember how many years ago I read this), but this seems implausible as we’d therefore expect caffeine as an adenosine antagonist to worsen mood.
The more I learn about sleep, the weirder and more fascinating the links with mental health become. I have a patient who was diagnosed with a non-epileptic attack disorder by neurology, but who is describing many symptoms of narcolepsy with cataplexy, enough that we have referred to a sleep specialist who can do sleep and awake EEG and maybe a lumbar puncture to confirm hypocretin levels. The patient is also reporting frequent nightmares relating to past abuse and trauma, and I am wondering if their hypocretin levels are normal that perhaps psychotherapy for the past abuse might alleviate the nightmares, help the patient to sleep normally, and correct the sleep paralysis and possible cataplexy they are experiencing.
I’ve had patients with PTSD who were experiencing recurrent trauma related nightmares in *every sleep cycle* awaking about once every 90 minutes to 2 hours. Simple trauma focussed CBT fixed this, surprising even me. I’ve heard other therapists and patients describing massive improvements in sleep post successful PTSD psychotherapy.
On the seasonal affective disorder link, I note that melatonin is synthesised from serotonin, but have not been able to find any anatomist to tell me if the melatonin in the pineal gland is synthesised from the supply of serotonin manufactured in the serotonergic neurons of the raphe nucleus or not, so this may be a red herring. Still, we might except a hypersomnic depression to more closely resemble an SAD presentation and a hyposomnic depression. Of course you also know that mixed depression and anxiety or atypical depression is often noted to occur more frequently than depression alone.
I have narcolepsy with cataplexy and it went undiagnosed for years, even after I went to a sleep specialist. In their defense, going over their notes much later I noticed they had scrawled in the margins that I might have narcolepsy.
For years I had no idea what was wrong with me but I knew *something* was. I'm in treatment now and doing so much better, life was getting borderline unlivable before.
I’m really pleased that your narcolepsy was diagnosed and that treatment is helping. I am unsure if my patient has actual narcolepsy or if the sleep paralysis is being caused by exhaustion due to impaired sleep in some way, hence the referral to the specialist. I will be very glad if the specialist can figure it out, as I guess you were.
Curious...what were the pre-diagnosis symptoms and what is the treatment that is working for you? I have frequent sleep paralysis and fall into immediate action-movie style vivid dreams upon sleep as well.
I don't have lots of sleep paralysis (though I have had it on occasion, not enough to be an impairment). I do have the hypnogogic hallucinations now and then. Really the only bad narcolepsy symptom I had/have was/is cataplexy.
The main reason my diagnosis was complicated I think was because I had been previously (correctly) diagnosed with Tourette's syndrome (and I have basically all the symptoms there, including the infamous coprolalia). Having one movement/behavior disorder on the charts caused pretty much everything to be interpreted through that lens, I feel.
In college I started to have really classic textbook cataplexy symptoms -- heightened emotional states or sudden noises would cause me to just drop limp and be unable to move for several minutes, while remaining fully conscious. I'm now treated with Xyrem, which works really great, but the government in their infinite wisdom decided that only one company should be allowed to manufacture it and that means they can and do charge monopoly prices (it's all an insurance racket though as if you can't pay they'll charge you a more normal rate).
Thanks. All the hoops one needs to jump through for a diagnosis...referrals, failure with other treatment, sleep study, etc. has never seemed worthwhile to me, but now that Xyrem is available that potentially changes the equation. Frankly, I'm surprised they managed to get it approved, even with all the special FDA rules around distribution, what with the hysteria surrounding GHB.
Xyrem, at least for me, has been magical. For my tourette's I was put through the typical pharmaceutical roulette wheel, trying a dozen different drugs, many of them off-label, trying to find something that would work, and all they did was make me gain weight, feel incredibly drowsy, depressed, and suicidal. I didn't have high hopes for pharmaceutical treatments for my Narcolepsy, but then -- whiz bang, here's this miracle drug with basically no side effects that nearly completely eliminates your symptoms. I really couldn't believe it. Not sure how typical the results are.
Xyrem has been pretty miraculous for me, too, even though I'm a decidedly noncentral narcolepsy patient (to the extent that several specialists have questioned whether I actually have narcolepsy.
There is also some suggestive evidence (https://pubmed.ncbi.nlm.nih.gov/11696070/) linking natural short sleepers to a kind of mild anti-depressed/hypomanic phenotype.
Possibly related: Narcoleptics supposedly enter REM state almost immediately upon falling asleep (talking like 5 minutes). I'm Narcoleptic and it definitely matches my personal experience in having wildly vivid and epic dreams even if I just doze off for 10 minutes. Does that have any further implications for these findings?
On the abnormally low confidence on predictions / overly strong or trapped priors theory of depression, it might not be as much a feature of the whole brain as of particular areas involved in reward, reward prediction, and reward prediction errors that are causally involved. Although the whole brain as a neural network could be using Bayesian prediction to predict inputs as per the Bayesian brain hypothesis, when it comes to depression (and perhaps anxiety) it could be regions such as the orbitofrontal cortex, amygdala, and the habenula that are affected in reward, reward prediction and reward prediction error.
I'm nervous about asserting that because depressive symptoms seem to affect so many areas - eg psychomotor retardation, appetite, cognition, autonomic regulation, etc.
I recently reread your posts on depression as networks, guyenet on motivation and the chamber of guf and I think they are all relevant to the topic of depression (but I'm confused about the sleep part). Also, this comment will be way too long for the top layer, so I'll add it here for easy condensation.
epistemic status: I'm an engineer and I think a lot about feedback loops and modeling, but I'm completely ignorant of most psychology and psychiatry. I'm currently thinking about this from a predictive-processing perspective, but this is an ongoing experiment and I'm not sure myself where this comment will lead me.
So let's try a modelling approach.
For sake of the argument, let's assume we're building a robot. It's supposed to be autonomously deployed to unknown worlds (minecraft, other planets, household robot for humans, whatever) where it has to find resources on its own and actions will have a cost (it needs a positive balance for energy intake, oil, ...); the environment will have periodic cycles (day-night; seasons). And it will not only be one of the robots but many, so they could cooperate or compete for resources, but there won't be any central coordination structure.
Now, let's think about what kind of structure we could give our robots that'll maximize their chances of thriving in the unknown environment. It would need some general possibility for building an internal map of the world around it, so let's implement to a predictive processing structure that generally tries to predict actions and minimize the free energy of it's internal model. Probably balancing various needs would be OK and we could do a random selection of task priorities comparable to the mechanisms described in the chamber of guf / guyenet on motivation posts.
Now: how active should our robot be?
The obvious answer is: We don't know! It's probably quite clear what kind of internal structures it'll need (vision, motorics, a possibility to harvest and use ressources, reproduce, cooperate / coordinate / compete with others), but without knowing exactly what kind of world it'll be deployed in (how many easy ressources vs. how many traps are there? how cooperative will be the surrounding agents?), there's nothing we have in the predictive processing toolbox to choose a straight-forward algorithm.
So let's make the best we can out of it:
- do a genetic algorithm that randomly sets activity thesholds in various robots: If it works well, there should be a way to pass this level on to more robots.
- do a medium/short-term feedback loop based on rewards: in a high-success-rate scenario, increase the activity level; in a negative-feedback-environment, decrease the activity level. If possible, tie this to obvious predictors (e.g. increase activity levels at warm temperatures if these are predictive of a high-reward scenario)
- include a slowly moving feedback mechanism that slowly changes the threshold over time; As we cannot tell how high/low the threshold should be, I assume we'll set an "expected number of activities per day". Each day we do more than intended, slightly raise the threshold to avoid more activity than the robot can handle (usually very high activity is very very costly). Set this to a very-sowly-moving timescale so this will build a homeostasis over months, but not override the medium/short-term feedback loop described above.
There are a few obvious ways something like depression could rise in our robots:
a) the robot just has bad luck and recieved an inherently very high activity threshold in our genetic lottery.
b) the robot's internal model is not calibrated very well and for whatever reason, predictions don't work as intended. Model calibration within a hierarchical model is really hard, so maybe there's something like to loose priors that make prediction of the right task at hand not work out as required by the current environment.
c) something really bad happened and pushed his feedback system out of the limits. Hopefully, it'll rearrange itself after some time, so unless the bad episode destroys his complete internal model structure of the world, it'll probably pull itself together sooner or later. (note that this could be a "big traumatic event", but also just an unintended prediction error, depending on this individual robot's calibration for how long-term it's planning strategy should be and how hard it's trying to maximise "always be right with your predictions" by having a really large feedback punishment for unexpected outcomes)
d) unintended feedback loops: This is an interesting one as there are various feedback mechanisms working against each other, so let's explore it a little bit:
Suppose something good happens, so the "be more active" feedback mechanism kicks in... so the robot will be more active, possibly getting more positive feedback, and so on. At some point, the slow compensation mechanism decides to raise the bar of "level of potential required for action to be taken". As a lot of good feedback is streaming in, this will override a slightly raised activation threshold, until at some point the threshold is raised really high AND something unexpected or bad happens. Now the stream of good results is broken and the activation threshold is high, so the robot will basically sit there and do nothing. After a long-enough time of being inactive, the activation threshold will finally be low enough that the robot takes some actions... leading to the next stream of positive events...
What does this mean for depression in humans?
a) and b) are interesting because they are "genetically inherited". I'm not even sure they are the same cause, but from the outside, it'll look the same, as the robot will have a lot of reluctance to choose action. I'm not sure if they would feel the same to the robot (if you could warp yourself into two robot's counsciousness, that is).
c) could be a really bad outside event (like losing a lost one) or a very traumatic experience (like being in a bad accident and losing a limb). In this case, the internal model would need a lot of recalibration (to accomodate both for the prediction error of not forseeing the traumatic event and the changed status of the world, e.g. no longer using your arm if it was damaged in the accident or having the lost person as a core part of your social structure). Again, I'm assuming that how bad this hits the robot partially depends on it's internal thresholds for sensitivity, but also of how robust the internal structure is right now: is the recalibration a feasible task or almost impossible as the event ripped out an integral part?
d) would look a lot like manic depression, cycling between very active and totally passive phases. It would also look a lot like known feedback loops, e.g. the lynx and hare predator-prey cycles.
From the outside, this would all look very confusing... like a large untangled mess where diagnosing one inactive robot, it's not always clear which clause is to blame for the inactivity and which button to press / measure to take to re-stabilize the robot (unless you have something like a complete debug access to RAM + source code). Also it would look a lot like the structure described in the network theory of depression, where many different symptopms and causes are related to each other. Oh, and unless you can directly access the sensitivity threshold (and all other relevant variables), the best thing you can do is maybe access one of the slow-feedback mechanisms and wait for the feedback to kick in after a few weeks.
What does this mean for predictive processing models of depression?
I'm not really sure, but to me, modeling depression looks like a tough thing to do, as there is no obvious "single path that fits all causes" and no clear evolutionary / modelling option that solves all the problems you run when trying to conceive a social robot or whatever. I'm not to sure what to make of this, but as a starting point, I'd be wary of simple, monocausal models for depression...
First, I am curious whether you mean "batch normalization" of artifiacal neural networks, or you actually mean "renormaliztion" which is something different and much more esoteric and unusual.
Batch norm does something like regularize neural network inputs so that all the inputs are rescaled into the same domain of possible values. A loose analogy would be your perception that an object "looks the same" whether it is illuminated by 10 lumens or 10,000 lumens; your eyes and brain have done a normalization for you across a broad range of objective input values.
I'll not from a machine learning standpoint that learning is all about leveraging nonlinearities. If you are routinely going through your network and just kind of smoothing everything out, then algorithmically you are probably making 99% meaningless changes that do very little and 1% terrible changes that effectively destroy some precise, interdependent arrangement of synapse weights.
It might be fun to take those creepy amorphous dog-generating GANs from a few years ago and intentionally do such an operation on them, "averaging down" all the weights, and then observing the result. I am not even all too sure what to predict. It is possible that, instead of seeing a frothing sea of Lovecraftian doggoths, you would get a more vague and less coherent visual image. It's also possible that you would still see some dogs, but maybe fewer. Or maybe you would see a disorganized mass of eyes and muzzles with no structure. Or maybe it wouldn't work at all.
I think this might be a reference to the DeepDream algorithm that "uses a convolutional neural network to find and enhance patterns in images via algorithming a dream-like hallucinogenic appearance"
From what you wrote, it sounds like you're most likely talking about regularization, which is where you periodically make all the weights (read: synaptic connections) in your network a little bit weaker. Renormalization is a thing in physics, but I don't know exactly how it would apply to neural networks.
This doesn’t match my idea of regularization - I think of regularization as a broad class of methods to reduce overfitting. I thought renormalization was a good term for what Scott’s talking about.
But also, batch normalization in NN training is thought to have regularizing effect - so there’s definitely a link between the terms! I’ve even seen a random article or two saying you don’t need dropout if you do batch norm, and dropout is a super popular way to regularize.
Did you mean that by the end of the day you have learned three synaptic weights, 0.9, 0.5, and 0.3, and through sleeping the brain does something like normalize them by their sum (1.7) to 0.9/1.7 = 0.53, 0.5/1.7 = 0.29, and 0.3/1.7 = 0.18? That way you incorporate the new observations and over many wake/sleep cycles in theory you dampen the noise because definitionally signal recurs and noise doesn't. I also would have guessed that the right verb is "renormalize".
And then the argument is that some kinds of depression learn fine but normalize lower than a reference bayesian agent would? And therefore symptomatically they diverge most from that reference bayesian in the morning but Central Limit Theorem brings the total synaptic weighting closer to the reference bayesian the longer the person is awake?
And it seems like there are other places you could lower the total weighting, which could look different over the course of the day. If a person learns with less confidence than our reference bayesian, then perhaps they could start chipper but start to drag over the course of the day.
One should be a bit careful to jump too quickly from GANs to brains. It's true that regularization is an important technique in machine learning, and it is fair to ask whether it might have the same effect in the brain.
But there is an important difference. Artificial neural networks are mostly feedforward networks. The brain is a recurrent network. If you forget normalization in a feedforward network, then you get overfitting and a larger testing error. If you forget normalization in a recurrent network, you get an epileptic seizure and die. Reducing overfitting may be a secondary effect of normalization, but the primary effect is "not dying".
Of course, there are some recurrent networks in machine learning as well. But usually the activation function is then designed in such a way that no seizure (uncontrolled explosion of activity) is possible. Strangely, the activation function of real neurons is not like this. It is normal for one neuron to fire at 100 Hz, but if a large fraction of neurons fire at 100 Hz simultaneously, you are close to dying. If excitatory neurons band together too strongly, the best you can hope for is a complete temporary shutdown of the brain area (there are some mechanisms which can do that).
I don't know why evolution didn't choose a different activation function, but the brain relies pretty heavily on homeostasis for not dying.
This, plus experiences of friends and loved ones, definitely makes me wonder if some depression-adjacent sleep deprivation is trying to hold on to/chase the feeling of increasing synapses? Especially because getting enough sleep is something I had always tried to help depressed people with. But Luke comments below, maybe the negative side of sleep deprivation is just the extra-long sleep phases that follow?
Anyway, we can help with the cabin in Alaska, if you're looking to put your six months in.
I think probably not. Most of the time depressed people go to sleep late, it's just because they don't have a lot of willpower or energy and it takes some willpower to make yourself go to sleep on time.
During a few depressive periods I definitely became aware that I was self-medicating with sleep deprivation. I’ve always been a night owl, but when I was depressed I would sort of ride the insomnia and the depression wasn’t as bad the next day. This could be because I had been productive in some way the night before. Or maybe I was just too tired to dwell on depressive impulses. Either way, it was not sustainable and the eventual inability to think straight would make me get some sleep, usually requiring a mild sleep aid.
Definitely a very interesting theory, but I'd be very cautious about buying into it. Just because staying awake makes you feel better and sleep makes you feel worse, doesn't mean sleep is doing something bad for your brain.
There's an analogy with muscles and sleep: If you work out really hard and then get a good night's rest, the next day you'll be sore. If you work out really hard and only sleep a few hours, you often don't get sore the next day. (That sounds counterintuitive, but it's a widely experienced phenomena.) But skipping sleep to avoid soreness is absolutely the wrong thing to do! The soreness is part of the recovery process. If you keep skipping sleep, your muscles will get weaker and weaker as they never recover from your workouts. So, by analogy, I'd worry that skipping sleep to treat depression might be trading off important brain recovery mechanisms.
Apparently Mr Trump holds some rather alternative theories (about having a finite amount of life energy) that have contributed to his minimal-sleep lifestyle.
This leaves me wondering if maybe he has built an unusually stubborn network of synapses that parallels his apparently astronomical levels of dopamine -- making him famous for his astronomically high levels of confidence-in-predictions. Resulting in a character who can absolutely never admit mistakes.
Lord, I wish there was more pruning going on up there!
Somewhat relatedly, the book "Why We Sleep" by Matthew Walker discusses studies that found that REM sleep reduces trauma: in sleep, the person re-lives traumatic memories, but in a detached way due to the emotional system not being fully connected. This causes the memories to be less emotionally-laden and traumatic going forward (or, one might say... "downgraded and renormalized"...)
Those studies might be relevant to the REM / non-REM question.
Thanks, that's good to know! Still, the book is unlikely to have made up studies on REM sleep and trauma healing -- so those could be looked up and evaluated.
Having read both the book and the critique, I'm not sure that much of the criticism actually lands. Translating academic research into a popular trade book is always going to involve some level of abstraction and simplification, and the criticism I've read seems mainly to nitpick the particular abstractions and simplifications Walker used when presenting the research.
Like (from memory here) there's a plot he shows with some particular trend. And he chose to omit one single datapoint which lay outside the fit. To me, this seems OK... It's exactly the kind of thing that would throw off a lay reader, while a trained scientist (who understands things like statistical variance, etc) will know it can probably be ignored.
I'm in science communication myself, and the type of things Walker chose to elide or gloss over seem pretty par for the course, if you're presenting research to the public.
Extremely speculative, wondering if others have the same experience:
I often find reading a few highly analytic longposts is the best and only consistent way I can start the day. It seems to actually get my brain "working" while I can't motivate myself to do anything else.
As per the post, the later the day gets, the more energy I have.
But I wonder if reading non-fic essays is more "stimulating" to neurons than say, making breakfast, or going on a walk?
Funny you mention this, but I've had a similar experience. On days I feel very non motivated to work (happens somewhat often), reading cool shit kick-starts my drive to do things.
Do anti-seizure medications sometimes called “mood stabilizers,” like lamotrigine and gabapentin for example, ever treat depression? If they are ever effective in treating depression, not just anxiety or bipolar mania, how does that relate with the parts of this post about seizures, if at all?
Is bipolar depression so different from unipolar depression that anti-seizure medication can treat it effectively whereas unipolar depression can be described as effectively treated in the opposite way?
If anxiety can be a symptom of depression, again, why are another kind of anti-seizure medications, benzodiazepines, so effective at treating it? Admittedly they are described as potentially causing or worsening depression. And often prescribed together with other medications.
This post made me wonder, why is it that anti-seizure medications are ever safe enough and even helpful to give to depressed patients?
But I think I can answer that for myself: there are thresholds, risks, sometimes opposing symptoms and conditions co-occur and need treating simultaneously.
What I am more curious about is: does any kind of depression ever look like a seizure disorder instead of something like the opposite?
Lamotrigine is effective in bipolar depression; its effectiveness in unipolar depression is kind of unclear, possibly leaning towards no (though I will very occasionally use it as a last option). I wouldn't make too much of the "seizures cure depression and antiseizure meds sometimes cure depression" paradox, everything has lots of different mechanisms of action and probably they're effective for different reasons.
It also seems relevant that sleep-disordered breathing is common and often goes undiagnosed, yet is often comorbid with depression. Is it possible that some of of the correlation is because people with sleep issues often feel worse when waking, and people with sleep issues are more likely to have depression?
It's kind of hard to find great numbers on this, since at-home sleep studies frequently miss sleep apnea and usually miss other issues, but according to random popular articles (Healthline and similar), about 40% of people with obstructive sleep apnea (OSA) have depressive symptoms, and possibly 15 - 20% of depressed people have undiagnosed OSA. Again, I'd expect these to be underestimates, since the one reference I clicked through (10.1016/j.jpsychires.2019.06.015) relied on really basic at home tests for half of their data.
To me it is crazy that testing for sleep issues is not standard practice for depression given that OSA and other sleep disordered breathing issues are common organic causes for depressive symptoms, and treatment (PAP, surgery, etc.) can improve these symptoms, but that's a bit of an aside.
Testing for sleep issues is hard. The usual process is to screen for them (by asking eg "do you snore?", "do you feel tired after waking?", etc) then test the people who reach some level of suspicion.
It's getting easier. With a Fitbit app, for example, one can view one's sleep on a minute-to-minute graph that pretty accurately tracks time spent in deep sleep, REM, light sleep, etc. So that's a good start (I typically get a horribly deficient 3-5 minutes total of deep sleep).
Then, if you add a modern CPAP (or autopap) into the mix, you can actually layer onto the same timeline a map of all your apneas -- and even discern whether they are likely to be structural or central in nature.
It's also true that Depressive symptoms sometimes look a lot like a sleep disorder, without one being involved. My wife is a sleep tech, and has gotten me two separate sleep studies plus an EEG because of my depression symptoms, because they were legitimately a decent match for sleep apnea of some sort. Everything came back negative though; it's sleep issues caused by depression, not breathing.
Depression in small children exists, but is hard to detect. Symptoms are too unspecific and the abilities to introspect and verbalize what they experience is in the range from missing to yet insufficient.
Sleep deprivation leads to ROS accumulation in/on the luminal-side of the gut but not other organs (including the brain). sci-hub.se/10.1016/j.cell.2020.04.049 This is evident in experiments with mice and flies after even two days of deprivation. Graphically evident with photos as in Fig 4. With longer deprivation, death results. If the luminal-side ROS is discharged through the by-mouth use of various compounds such as NAC, melatonin and lipoic acid, the animal is rescued from death even with sleep deprivation. Recovery sleep dissipates ROS accumulation in the gut. The effect of sleep deprivation is not all (or ?even?) in the brain.
The gut microbiota influences the brain and behavior through, among other things, the vagal nerve. sci-hub.se/10.1016/j.neuron.2019.02.008 Preventing vagal communication between the GI tract and the brain increased occurrences of psychiatric-related disorders. Vagal nerve stimulation can substantially improve MDD. Gut bacteria also influence the brain and psychiatric disorders through the vagal nerve. The vagal nerve has about 80% afferent (gut to brain) and 20% efferent (brain to gut) fibers.
Hypothesis: If the depression is caused by a specific gut ecosystem signaling, suppressing the signal between the gut and the brain through ROS accumulation in the gut could suppress the depressive symptoms. As demonstrated in the first reference, sleep dissipates the deprivation induced ROS. A possible test of this hypothesis would be to have patients whose depression are relieved by sleep deprivation take one of the tested supplements (e.g. lipoic acid, NAC, et.al.) and test for continued depression relief with sleep deprivation.
Thanks for pointing out the connection between sleep deprivation and gut ecosystem (which I hadn't thought about before)! In particular, since "if the ROS[the thing that accumulates during sleep deprivation] is discharged..., the animal is rescued from death even with sleep deprivation", this makes me think that depression might be a kind of way of being alive, because becoming less depressed by staying up eventually kills you? (But I also notice that non-depressed people typically stay non-depressed after going to sleep for a night and waking up in the morning, contra an intuitive model that of sleep & depression in which each sleep is a Big Depressing Injury, so there must be something about depression that makes sleep depressing for depressed people but non-depressing for non-depressed people**.) Either way this sounds like a beautiful and highly plausible theory. Also since NAC/melatonin/lipoic acid clear ROS, this suggests that either those or their pharmacological opposites should be depression treatments.
*Is this not allowed anymore as a form of pseudo-upvoting? If so, sorry and please ignore.
**And now I've reached semantic satiation for words that begin with "depress". Congratulations.
I guess this theory can explain your problem of trapped priors: Whenever a small not-so-negative experience (eg a only slightly aggressive dog) happens it gets deleted by a faulty renormalization afterwards and only a clear positive experience (eg a cute puppy in a cage) can help. Apparently, a correct renormalization would keep experiences of the first kind, but for depressed people it is more likely to throw small updates away (since their brains have less connections overall).
Maybe I'm melancholic depressive, though I don't know what that means. Any time I wake up at maybe 6-8 am, I'm suicidally depressed. The first thing I think on waking is, "Dammit, I'm still alive." But I know the feeling will dissipate in an hour or two, so I try to ignore it.
This happens regardless of my sleep cycle. If I go to bed around 3am and get up between 10 and 11, I have no depression at all.
(I haven't noticed whether it varies with season. I don't have natural optimal sleep-wake times; I have a natural daily cycle of 25 hours, so my sleep cycle moves forward 1 hour each day until I force myself to get up early and suffer a sleep-deprived day to restart the cycle.)
~25 hrs is normal in humans, it is reset daily by light cues. A strong light to wake up with (see the Lorien post on light therapy) could be worth a try if you haven't already.
If 25 hrs is normal, then humans must have evolved 200 million years in the future. Probably the Doctor was involved.
Thanks for the pointer, but I'll probably just keep doing what I'm doing now: always sleeping until after sunrise. I wonder whether Seasonal Affective Disorder isn't seasonal at all, except as an artifact of clocks. Just a bunch of people who get depressed in the morning twilight, but notice it only in winter because nowadays we get up at the same clock hour year-round rather than getting up when the sun rises.
Thanks for bringing this up. I was also under the “natural is 25 hrs” misapprehension, and went to look at https://en.m.wikipedia.org/wiki/Circadian_rhythm#Humans to look it up and paste here, and apparently that study was bad and after they redid it it’s much closer to 24 hrs
This corresponds precisely to my experience. I never thought about this systemically before, but I have always struggled with getting out of bed in the morning due to an overwhelming feeling of melancholic depression, exacerbated by the fact that I stay up later than would be wise for my own good because I usually feel most at peace with the world and therefore energetic at night. The occasional time I manage to stay up all night, I get to work the next day with verve and a more positive outlook, partly because of a feeling of accomplishment I suppose. Unfortunately there does not see to be any apparent solution to this, seeing as eventually Morpheus does call for his due.
It looks like a few people here are relating this blog to the one on trapped priors. Eric Hoel has some work (e.g. https://arxiv.org/pdf/2007.09560.pdf) on "The Overfitted Brain: Dreams evolved to assist generalization" that seems relevant. His theory (which he presents some data in support of but I don't have enough knowledge of the field to know how convincing it is) is basically that the brain, like a neural net, is prone to overfitting based on the data set that it encounters. And in neural nets they often solve that problem by adding noisy or corrupted inputs, which make the fit worse on the existing data but make the system overall more capable of being generalizable and adaptable. And that dreams are the human equivalent of that. From the abstract: "Dreams are a biological mechanism for increasing generalizability via the creation of corrupted sensory inputs from stochastic activity across the hierarchy of neural structures. Sleep loss, specifically dream loss, leads to an overfitted brain that can still memorize and learn but fails to generalize appropriately." I don't think I can quite put my finger on the model that connects them (maybe because it doesn't exist) but there seems to be a strong common thread between depressed brains suppressing new input, and dreams purported function in preserving openness/flexibility, and sleep being messed up in depression.
I wonder if this explains my experience, which is that I feel like I undergo a steady uptick in anxiety the longer I stay awake, which I think is a fairly general phenomenon, ie "your brain at 2am suddenly bombarding you with restless thoughts."
Yes. On the first deep inhalation (the "now I'm awake" moment) anxiety explodes (on bad days). It can take hours sometimes to come down and get functional. After falling asleep again it can happen again, even if the first waking up has been OK. I tried to see if that changes but up to four times (couldn't get back to sleep more often) it comes back.
So what does waking up around 3AM, often associated with depression, do in this model? Is some failsafe processing in the brain detecting "too much" synapse clearing and waking you up to stop it?
>unsurprisingly SSRI antidepressants decrease REM sleep a lot (not just in depressed people, in >everybody).
I found this pretty surprising and would be interested to hear more. How significant is the decrease? how dose-dependent is it? etc.
Background:
I have had insomnia for basically my entire life which could charitably be described as "severe" and uncharitably described as "Holy F****ing H*** how are you still alive?!?!" I take a few different meds and supplements for it, in addition to sleep hygiene and other lifestyle interventions, but the main medication is Trazadone, an SSRI, and when my sleep specialist first prescribed it after my sleep study (which was apparently pretty horrific looking, even to someone who dealt with severe insomnia cases every day) she said it had been shown to help produce deeper more restful sleep, rather than merely helping with sleep onset. This was important because a big part of my problem was not getting any rest even when I theoretically slept; my sleep study showed (IIRC, it was almost 20 years ago) something like 5 hours 'asleep' out of 9 hours in bed, only about 20 minutes of that REM, and zero slow-wave sleep, I was just staying in the first stage of sleep all night, when I was asleep at all. No one sleeps well during a sleep study, you are in a strange room with electrodes taped all over you, but this was bad even accounting for that. I guess I am rambling quite a bit now, but my point is I was very surprised to hear SSRIs would /reduce/ REM considering what my sleep specialist said back in the day, and how well it seems to help me given how I sleep when I am not on it.
Oops. you're right of course, it is one of the older (non-selective I guess?) serotonin re-uptake inhibitors. For some reason I was remembering it being an early SSRI. But if the decrease in REM is caused by increased serotonin wouldn't you expect the same effect? I'm afraid I've been on it long enough I've forgotten most of the reading up on it I did initially and have regressed to "as long as it keeps working I'll keep taking it"
I might be wrong about this, but I don't think trazodone reuptakes serotonin at all. I think it is mostly just an agonist or antagonist at various serotonin receptors.
I'm seeing it classed as a serotonin antagonist and reuptake inhibitor (SARI), but not much on the relative strength of actions, so it is possible the reuptake inhibition is simply insignificant.
Low confidence in predictions may be an accurate read of reality in the face of a stressful or unpredictable environment. In this way it could be an adaptation to stress which could also include social withdrawal, apathy, and pessimism, or seen differently a switch into a mode of conservation and survival as opposed to a mode of exploration and expansiveness.
The amount of sleep that people have goes down with age, until retirement when it starts going up again.[0] Definitely some of that is due to the amount that they're able to sleep (a person with a job and two kids is going to have very little sleep), but is any of it related to different needs?
> Consider this observation: high-income people spend less time sleeping and less time watching TV than low-income people. Why is this? “That’s obvious,” you might think: “lazy people spend their time sleeping and watching TV. Since they don’t work as much, they don’t have a high income.“
> Is there another explanation? Hamermesh suggests this one: everyone has to spend their time - we all get the same 24 hours each day. High-income people have the option of spending it on expensive things, like eating at nice restaurants and working out at fancy gyms. But low-income people can only spend it on cheap activities, like sleeping and watching TV. Put another way: if you have more resources, you face higher opportunity costs.
> The time use data allows for these theories to be distinguished. Hamermesh looks at people who have lots of income because they work (i.e., they have a high-paying job) and people who have lots of income for some other reason (e.g., their spouse works a lot, they won the lottery). And he finds that the second theory is better: the high-income people who don’t work aren’t spending much time sleeping or watching TV. So (on average) it’s high income -> less sleep/TV, not less sleep/TV -> high income.
This pattern - high sleep in youth and retirement, low sleep in prime earning years - could match Hamermesh's view.
That chart explains nothing about what's going on because it doesn't distinguish between weekdays and weekends.
Of course, some people get to sleep as much as they need, but I would estimate that a lot more people are chronically underslept on weekdays because of work and kids. On the weekends the underslept people try to catch up on sleep but do not always succeed.
I wonder if they have a more useful chart somewhere. I couldn't find it.
Has anyone ever tried sleep reduction therapy to treat depression? If avoiding sleep entirely seems to work until you go back to sleep, what happens if you only sleep 4 hours a night? How long can a human go with artificially reduced sleep?
What about going the other way? If we make people take sleeping pills to sleep more, do they develop depression?
For both questions, does changing sleeping patterns eventually cause the brain to renormalize to that level of sleeping?
Having been in the military in an active combat unit in an active combat role, even just for training purposes, the answer to "how long" is a very long time with sufficient motivation. I can't say the effects are in any way positive, but granted, there are a lot of confounders. Even when it's just Ranger School and you're not actually in danger, there are plenty of sources of stress aside from just the lack of sleep.
There are the ultra rare cases of people who just genetically need less sleep, including the famous Al Herpin: https://en.wikipedia.org/wiki/Al_Herpin. Wikipedia seems to not believe his claim to have never slept, but if he were telling the truth, would he be the happiest man in history?
Sleeping less than my body wants to helps with mine. 4h would be excessive, I think I average 5-6h recently, while the "natural" amount would be around ~9h.
Even if I crash on a weekend day as the body wants to catch up with the sleep debt, that's one day with depression vs an entire week with depression.
Perhaps it is an under stimulation problem? You sort of allude to this in the post, but do not really explore that possibility further.
I basically fit this description to a tee. I have some intermittent problems with depression. And sleep deprivation removes it completely. Periods of slowly slipping into depression followed by sleep deprivation to snap me out of it used to be a recurring pattern for me. But I notice that when I force myself to get seriously mentally stimulated (reading some difficult texts, doing math problems), the depression goes away. Or is less severe. Especially when done earlier in the day. Caveat though, it is not major depression for me, although suicide has occasionally occurred to me in the past.
And actually working too hard, spending too much time and mental energy during the day gets me into mildly manic moods where I have a hard time falling asleep. Usually followed by tiredness where I spend less time doing difficult tasks, where I slowly slip into depression again.
This is so effective for me, that when I get depressed I know I have been lazy and need to improve my work ethic. It is kind of annoying that this only works for tasks that I do not easily pick up, especially when depressed. So once I slip into a depression it can be hard to get out of it without sleep deprivation.
So each person might have different needs for mental stimulation and sleep. And perhaps sleep deprivation to snap out of it, followed by a constant diet of doing mentally taxing and rewarding tasks is the cure for people who suffer from this. It helps if you set an overarching and at least somewhat ambitious goal that you can get excited about.
It seems like a reasonable inference from this is that someone should try treating depressed people with small doses of melatonin, like it's a sleep phase disorder. Does anyone know if this has been tried? It's in no pharma company's interest to find out if melatonin can treat depression so probably not, but maybe that fancy patented melatonin that there was an SSC post about once.
Also, what about depression with atypical features? It's definitely not rare. So it seems like there are also a lot of depressed people who feel *better* after sleeping, and I wonder how that works with this theory.
Phase shift with light therapy (and sometimes melatonin) is moderately effective for depression. A recent study showed the combination of melatonin+buspirone seems effective too.
I don't know how atypical depression fits in here.
If this type of depression makes you wake up too early, do you have people take melatonin first thing in the morning (like you suggested for advanced sleep phase disorder)?
1. "You can't renormalize while the network is running." Says who? This seems like it might be a case of unconsciously extrapolating from *human* designed machines to natural machines. Human beings, for reasons of design and construction efficiency, do indeed design machines that generally need to be shut down in order to be worked on -- we have to close the road to repave it, we have to shut down the computer to replace its graphics card.
But I would say as a rule biochemistry tends not to operate this way -- there tend to be many multiple overlapping and redundant paths for getting the same, or similar things, done, and it appears pretty common for the body to work on a system at the same time as it operates. Bones don't grow longer (in children) only when they are not bearing weight, we don't heal wounds and injuries better or faster when we don't use the injured area, and we don't always choose to run only catabolic or anabolic pathways for the same substrate.
So I would say this idea should actually need to be measured to be sure it is true, especially since it's a key assumption underlying the idea that retuning the synapses needs to be done during sleep.
2. Why is it said depression gets "better" when one is sleep deprived? Maybe sleep deprivation merely adds a temporary cover layer of mania that hides the depression, as if you gave the depressed person a massive dose of amphetamines. That is, the change may not be in the direction of cure, but in the direction of adding *another* layer of brain dysfunction that conceals the original layer somewhat, so, away from a cure. If so, the last thing we'd want to do is try to extend it.
3. If a *general* synapse realignment underlay depression, why wouldn't there be much broader associated brain dysfunction? That is, shouldn't depressed people also have noticeably lower IQ, poorer executive function, worse memory -- even compromised sensory abilities and downgraded motor skills? That is, shouldn't an across-the-board physical degradation of the brain have across-the-board degrading effect on brain function, from soup to nuts? Why would we see the primary symptom in mood (which seems a fairly high-level complex epiphenomenon of neural function) rather than in, say, memory impairment or crappy motor skills, something more basal?
Depressed people do have lowerIQ, poorer executive function, worse memory, compromised sensory abilities, and downgraded motor skills (look up eg "psychomotor retardation", "pseudo-dementia", and the research on depression and olfaction). It's usually pretty subtle, but can be obvious in sufficiently severe depression.
Shouldn't it be quite hard to tease out the effects of simply being less motivated to take tests and succeed at motor tasks from actually being physiologically worse? I can see testing purely automatic reflexes for sensorimotor response (but not control) but can't even think of how you would try to compare intellect across test takers trying to do well and test takers who aren't trying to do well. Memory maybe, but how much is worse memory and how much is paying less attention because you don't care enough about what's happening to want to remember it?
"Maybe sleep deprivation merely adds a temporary cover layer of mania that hides the depression, as if you gave the depressed person a massive dose of amphetamines."
I'm inclined to see that as the way it works, particularly since the depression comes back once the sleep deprivation stops. The sleep deprivation quoted isn't merely "skip a few hours sleep", it's "long enough to start bouncing off the walls" periods.
On point 2: I'm thinking that in some cases, "a dash of mania" might just be the same as "a reduction in depression", and in other cases not. Depression is heterogeneous, and some depressions look and act more like mirror-mania than others. There's a type of bipolar episode called a "mixed" state, where symptoms of mania and depression co-occur--so there are at least some cases of depression that aren't cancelled-out or masked by manic symptoms. In these cases you see a lot of agitation, anxiety, irritability. I have to wonder what kinds of effects sleep-deprivation has on the 30% of non-responders. (Interesting to note that treatment-resistant depression may be stealth bipolar in a lot of cases, and you'd expect a population like that to have more sleep-deprivation responders than in depressives broadly, where I want to say the rate is more like 50%.)
I think there's something in that. I've heard they tend to produce less melatonin naturally.
My daughter has an ASD diagnosis. She used to be very troubled at school, and also slept badly. Since she started taking melatonin, her sleep has improved a lot, and she's had a massive reduction in meltdowns and challenging behaviour, to the extent that I'm not sure she'd get a diagnosis now. (She still has the "good" or "neutral" autistic traits, like systematic thinking, good memory, and preferring her own company.)
Speaking for myself, I don't know whether I'm on the spectrum or not (several traits, genetic link, but never been formally tested), but I know that if I don't get enough sleep I'm much more prone to sensory overload, so I'm guessing her experience is similar.
Yeah, this aligns with some of my own anecdotal experience; I slept horribly as a kid and my autism symptoms improved substantially in my mid-20's around the time that my sleep improved too.
There are some ML failure modes this brings to mind as potentially useful ideas here.
Sparse gradients and local minima effectively make improving a model impossible. In practice, this usually means you throw the model away in start over. But you can't do that with a human, so you need some way to escape from the local minima. Stronger gradients are one way to potentially do that - a steeper gradient will tend to push what would have been smaller updates into bigger ones. The downside of taking bigger steps like this is that you often end up in a worse place than you started, because bigger steps means that the local gradient is less predictive of how good your new state is going to be.
But if your problem is escaping a local minima, that can actually be a very good thing! Even if your immediate situation is worse, you *might* now have a path to improvement, since even if your new state sucks, *it might be next to a place that doesn't suck*, and that might be next to a place that sucks *even less*, and there might be a path through state space all the way to an actually decent model.
More generally, a lot of depression treatments looks awfully similar to the kind of desperate tricks you might use to get a model out of a local minima.
"More generally, a lot of depression treatments looks awfully similar to the kind of desperate tricks you might use to get a model out of a local minima."
SSRIs and bupropion both effectively increase the signal provided by various neurotransmitters. Assuming that those transmitters are involved in the feedback loops that do some kind of gradient descent in biological neural networks, more signal = more gradient = bigger steps = better shot of escaping local minima. Sleep deprivation, in addition to the mechanisms you describe in this post, also just breaks a lot of feedback mechanisms things - usually bad, but great if you just want vastly increase the variance in feedback to escape a local minima. Electroshock therapy seems to basically be throwing a ton of (nonmeaningful) signal that knocks things out of whatever steady state they're currently in and kicks you some ways away in state space to some point that may itself suck, but has a better shot of having a gradient you can descend. Exercise generates a lot of signal. Changing your environment generates a lot of signal. Social interactions are complicated and signalful. Psychadelics radically alter whatever state your brain's in - and wherever it ends up, it's less likely to be a local minima than wherever you started.
The general testable prediction here is "increasing feedback to generate steeper feedback gradients or otherwise significantly altering the state the brain is in, pretty much no matter how you do it, has a chance of escaping the local minima of depression. More intensive feedback/more aggressive state shifts are more likely to be effective than less aggressive mechanisms. You're not aiming for improvement - just change to a point where improvement may be possible."
Sorry, you probably wanted me to elaborate on the ML side of this. A lot of ML failure modes center around sparse gradients - basically, gradient descent failing because all of the reachable points in state space are worse than where you are. You're in a valley in N dimensional space, you can't go downhill.
In industry, the usual practice is to throw away the model and start over. But throwing a ton of signal/increasing the gradient to explore the surrounding space/randomly perturbing weights are the sorts of things you would try if that wasn't an option. You can sort of see this in guides on how to avoid e.g. mode collapse in GANs: https://github.com/soumith/ganhacks . A lot of the advice is based on the idea that you really want to keep generating some kind of signal, and noise and randomness keep popping up as mechanism to induce that.
If it helps (Scott, not you, as you obviously understand this quite well), I think you can understand how dimensionality contributes to this by obvious analogy with normal space. If you constrain yourself to moving in a perfectly straight line, your options are pretty limited in where you can go. Open up to a second dimension and suddenly you can rotate 360 degrees and have infinitely many options just in the choice of direction. Open up to three dimensions, and well, think of how volume scales compared to area. Suddenly, the search space is a lot larger. This extends quite intuitively to extremely large dimensional spaces. The search space is absolutely unimaginably enormous and the fact that we find reasonably good solutions anyway is nothing short of a mathematical miracle.
This is exactly why regularization works so well. When you have so many dimensions of data, the overwhelming majority of them are insignificant, and the Bayesian optimal search assigns little or even zero weight to most of them.
I have no idea if this sort of observation analogizes in any way to human brain anatomy, though, as it suggests a good treatment strategy when the brain goes wrong is to just willy nilly destroy synapses at random, figuring most of them aren't doing any good anyway.
I mean, the sort-of-surprising-thing in this context is that you can have 500 million dimensions and still have basically no slope (either a local minima or a flattish valley). This make sense when you switch back to thinking about it as an optimization problem, because intuitively reasoning about high-dimensional spaces is hard (e.g. high-dimensional oranges are almost all rind)
> I mean, the sort-of-surprising-thing in this context is that you can have 500 million dimensions and still have basically no slope (either a local minima or a flattish valley).
It makes more sense if you stop thinking of it geometrically and start thinking of statespaces.
If I have a processor with 64MiB of memory... sure, it's got a state space of >500m dimensions, but you can still easily come up with local optima for a lot of objective functions.
For one example... if we're using our 64MiB of memory as the input to an optimizer that's trying to successfully execute an instruction at the highest address possible, a nop slide across all of memory is a fairly obvious local minima. You can do better, by remapping some physical address to the end of the virtual address space and executing it, but that requires setting up page tables / etc / etc, none of which is going to help you in this local state.
Of course this replaces one question with another (why do things start looking statespace-like?). And glosses over discrete versus quasi-continuous variables.
Has anyone tried just putting a depressed person into a really exciting or unusual or terrifying/thrilling (but safe) situation? Like taking them to an amusement park and making them ride on all the scariest coasters and rides?
Families try this a lot, this is the standard family let's help out the depressed guy strategy, especially before diagnosis but even after diagnosis, families usually will keep trying this periodically.
In my experience, this has diminishing returns, that is, if someone has not had something enjoyable for a while, the impact of having an especially enjoyable and exciting experience can be significant. However, if the depression is persistent, trying again and again to throw enjoyable experiences at someone eventually becomes irritating and exhausting and reinforces a feeling that really enjoyable experiences are out of reach. In really deep depression, sometimes the depressed person will barely be aware of the enjoyable experience even as they go through the motions, since their attention will be on the unrelenting drama in their skulls.
In short, it's a good idea, worth a try, but not a sure-fire cure.
I hate heights, I've been on a Ferris wheel once in my life and that was more than enough, and if you forced me onto a roller coaster not alone would I come out still depressed, I'd come out depressed *and* unmercifully pissed-off so I hope you have a good dentist because you'll need them because I'm gonna knock your teeth down your throat
My wife is depressed (on treatment and doing fine) and she *loves* roller coasters. Days we can go to amusement parks rank among her favorites. She's speculated that if she could just ride roller coasters on the regular, she might not need Zoloft.
If any strong, random perturbation has some reasonable chance of knocking someone out of depression, you have you wonder why more people don't just stumble on depression cures going about life, getting almost hit by cars or tasting ghost peppers for the first time. A lot of depression does spontaneously remit, so this could be happening, but my thought is that within a depressed states, just submitting yourself to the right kind of stimuli is not going to be enough to get you out of the local minima--the problem is that you aren't interpreting signals well, not that you aren't getting getting the right kind, and everything is just more depression-fodder. A signal strong enough to shock you out a local minima depression would have to be something that the depressed model of the world just was not able to account for or incorporate--otherwise you just roll back down the hill, no?
Rather than stronger signals, I think the commonality in depression treatments under the local minima theory would be things that alter the updating process itself. I think that would explain the lag between treatment and improvement, actually--it takes awhile to slowly grow out of a maladapative network, and changing the weight updating procedures just kicks that process off. If a person were just settling into a new pattern after a strong shock, you'd expect rapid improvements instead.
The reason I asked about roller coasters is because the "any strong shock" theory does run true to my experience. Granted it is just anecdote, but the one time I suffered from depression, a completely random and surprising experience *did* seem to immediately knock me out of my depressive state. I was sitting on a bench, literally thinking to myself how I really wished my life would end, but didn't have the energy to be bothered to do anything about it, when suddenly a very, very old friend who had moved far away, that I had not seen in years and never expected to see ever again, happened to walk by. It was completely shocking and surprising, and after we had a brief conversation exclaiming about how weird it was, I found that I felt 100% back to "normal" and no longer depressed. Now, this depression had only lasted for about a month, and seemed to be situationally induced after I had been sick and forced to stay in bed for several days. So it was not an ongoing chronic depression, and perhaps wasn't that "deep", though it had been unrelenting for a month. But in that case, after the shock occurred, it was like I was literally jolted into a different frame of mind and back to normal. I felt instantly better and the depressed feelings didn't return. So the strong stimulus theory rings true to me, at least for more minor or short-term depression. Of course, it's not easy to manufacture an artificial but effective "shock"....I thought a stimulus that ramps up adrenaline and emotion, like a rollercoaster, might work.
I think we actually agree a lot more than you think. The word "strong" is doing a lot of work here, and I think unpacking it dissolves some of the apparent disagreement.
A common way to stumble into local minima is with vanishing gradients - coefficients so near to zero as makes no difference. You might end up in this situation if the reward feedback mechanism for a given system consistently couldn't make use of some piece of information and so updated to treat it as noise. If that happens too often, eventually the system is going to treat everything as noise. And once that happens, you're stuck. To pick a more specific failure more: If the first layer of your network just multiplies all of the input by 0, the rest of your network can never learn anything.
To a network beset by aggressively vanishing gradients between the input and most of the network, what most systems would react to as a strong signal would elicit the same response as a weak signal.
So, you can try perturbing the weights, forcing the network to start processing information again, and hope that this time you get a little luckier and it finds some way, even if it's small at first, to get some reward out of some of the input. Or - maybe there are input channels that are usually zero, so the weights escaped the great purge of noise reduction that killed everything else. Parts of the network that never played a significant role in generating the output that the loss function penalized so aggressively, and so never got back propagated into oblivion.
If you could generate some input on those unused channels, and the non-zero parts of the network find some way to marginally update the weights that improves performance - well, then you just might be in business. You can start descending some gradient. And then, the nodes that take input from the used and unused channels - well, maybe it turns out that some of them actually can make use of some of that previously-zeroed-out input, so it updates the input weight from there from 0.0001 to 0.0003, and then maybe 0.001, and then it turns out that you can combine that with some other input to improve performance a little more...
In the hill-descending analogy, this is equivalent to thinking that you're in a valley, and then suddenly your third eye opens and you realize that even though north, south, east, and west are no good, east-by-wourth-blorgward *does* go downhill
Electroconvulsive therapy seems a lot like this. It's effectively scrambling all of the weights and resetting network state. You can't really reset a human without inflicting actual brain damage, but we certainly try.
Yeah. I wish I knew more about the details of what ECT actually does to the brain, but from my layman's perch it looks a hell of a lot like aggressive random weight perturbation.
Having been through ECT, what the long-term effects felt like to me was having the connections between memories disrupted. (As well as the connections between aspects of what one might consider a single "memory".) It's been about 14 months since I stopped, and I'm still getting odd deja vu. My impression is that most or all of the data is still there, but how would I know? Sometimes I look at things I did or wrote in the past, and I can't connect to them at all. And then sometimes some other seemingly-tangential event will bring back some of those memories, memories that I didn't get from simply seeing the thing itself. And there's always the nagging worry in the back of my mind that I'm just fabricating the "new" memories on the fly.
The short-term effects of ECT were like all the pleasant parts of being drunk, without any of the unpleasant parts. It was quite nice. I'd come out extremely happy and smiley and dopey, and open and trusting and joyful to see whoever it was who came to pick me up. Then over then next 24-36 hours, I'd slowly start to re-connect with all the reasons I was depressed. My go-to literary analogy is from Bujold's "Komarr", when Ekaterin goes under fast-penta, and then comes out of it.
R. Pirsig described the effect of excessive ECT, an erasure of the personality and the subsequent encounters with the ghosts of memories, in "Zen and the Art of Motorcycle Maintenance".
In case you have read it, does his literary description match your experience in quality (obviously not in quantity or intensity)?
I read it a while back, but it's now on my to-re-read list for obvious reasons. :-) From what I recall, the fictionalized depiction in the book doesn't match up at all with what happened to me. I think my personality is largely the same, but the ECT got me out of a severe paralyzing depression. You know that thing in depression where thought kinda spirals around a black hole, and anything different you can think of immediately relates back to the depression somehow, and just drags another chunk of your life into the pit? It feels like that effect got disrupted, along with a lot of the connections in my memories. I think I'm better at holding on to the good bits, now. If that makes sense?
However, there are a lot of variations in how ECT is done and has been done, and some of the changes have been in direct response to problems people had, or to abuses like the ones depicted in the fictional "One Flew Over The Cuckoo's Nest". The current is applied to different parts of the brain, there's different types of current, and there's also a lot of attention paid to the frequency of treatment. The staff were the absolute nicest I've seen at a hospital (not that I have that much experience, thankfully). Even the modern name "electro-convulsive therapy" was chosen to move away from the word "shock". So while my experience in 2019-2020 was different than what was in the book, I don't think that has much connection to whatever actually happened to Pirsig in the 1960s. Maybe his experience was a lot closer to what he wrote.
> " newly published biographical information on Pirsig from Mark Richardson (Zen and now: on the trail of Robert Pirsig and the Art of Motorcycle Maintenance. New York: Knopf; 2008) has documented that the role of ECT in ZAMM is a 'literary device', added at a late stage in drafting the book. In reality the ECT had erased some short-term memory, but Pirsig's long-term memory had quickly returned. Richardson obtained this information from Robert Pirsig's (then) wife, from his sister, and also from his friend John Sutherland (who appears as a character in ZAMM). It seems that one of the most famous depictions of ECT, one that had appeared factual, was actually fictional."
So, not fully fictional, just massively exaggerated.
More generally, my guess is that ECT, like hypnosis, 'truth serum' (and other candidates?), are (or were) understood badly enough by the general public of their literary and artistic heyday to be an easy tool for authors to create a thrill, as a means for the (usually evil) adversary to exert extraordinary power.
What do you mean by this? After ECT most people eventually recover and have the same memories as before - why would this happen if you changed all the weights?
I realize that you weren't replying to me, but... what do you mean by "most"? I haven't done any formal review of the literature or anything, but my impression was that permanent memory loss was a common side effect, affecting up to 1/3rd of ECT patients at a minimum. (Varying depending on lots of things, including the exact type of ECT treatment used.)
So there's a few different things this brings to mind, which I'm going to list off in no particular order:
1. You are changing all of the weights, but it's important to remember that it's a perturbation, not a totally random reassignment. State space is Extremely Very Very Very Very Very Vast, and in the grand scheme of things you haven't moved *that* far. That means that *most* of the information you had before is still there.
2. Human brains and many large ML neural networks often contain a lot of redundancy. You can observe this in action with "knowledge distillation", the umbrella term for methods of closely approximating the results of a large, expensive model with a smaller, cheaper one - even though the distilled model contains a fraction of the raw information as the original, it still seems to "know" about as much. Most of the big NLP models you've heard of, like BERT and GPT-2/3, have distilled versions that perform very very close to the big version using a fraction of the resources.
3. Aggressively perturbing a model is likely to break it in some ways - this is the price we pay for our local minima escape attempt. But the things that *worked* about the local minima are likely to be recovered in short order via gradient descent. They're relatively close in state space, and it's likely that following the gradient from our new perturbed location will converge with our previous local minima along many dimensions (meaning a lot of the weights are likely to return to very near what they were before we shook everything up). The hope is that *some* of the dimensions diverge such we don't end up back in the same valley we were in before, and we can continue descending the gradient without getting stuck on a local minima.
4. I'm guessing that some machinery involved with recovery memories gets damaged by ECT, but that this is the sort of thing that's likely to get recovered by gradient descent (since successfully remembering things is "rewarded").
Scott's is an interesting theory but it does not hold. People with schizophrenia also have lower synaptic density (see https://www.nature.com/articles/s41467-019-14122-0 ) but so far nobody noticed that sleep deprivation helps with schizophrenia.
Secondly, my guess would that the negative correlation with the depression score will turn out to be spurious when we get a publication with a larger sample. The sample in the article Scott quotes is very small (n=40).
Schizophrenics also have very disturbed circadian rhythms, and the negative symptoms of schizophrenia can look a lot like persistent depression and are very difficult to treat. Since you can't treat negative symptoms in schizophrenia very well with other depression interventions, I wouldn't take the fact that an intervention works for some depressed people but doesn't work for schizophrenia as a sign of the goodness of fit of a theory about the etiology of depression.
On a lighter note, psilocybin increases synaptic density after a single administration (in pigs) https://www.mdpi.com/1422-0067/22/2/835 - Looks like we have a new therapy for schizophrenia and depression. And also you do not need to sleep, if you take some psilocybin, if Scott is right.
That's really surprising to me, as I would have thought psychedelics released us from all those well-structured connections -- much like the dream state.
The #1 most common side effect of TMS depression protocols is extremely vivid dreams, by a long shot. I have had one of these protocols and can attest, it's friggin crazy. If it's non-REM sleep that's doing harm, maybe TMS increases not only the intensity of dreams but the amount of REM sleep.
Most antidepressants (in multiple different classes) also frequently cause extremely vivid dreams, especially early on in treatment, so I wonder if there's some similar sleep mechanism somehow
Gina actually described three stages, REM, deep sleep, and a "transition period".
Deep sleep is the physical repair time, when the brain is least active.
REM and the transition period both are connected to the rewiring of your synapses, but one strengthens new connections, the other clears out unneeded connections. Or one organises, the other files/dumps depending on the results of the organising. Figuring out exactly what happens is obviously an area of ongoing research.
Two interesting notes that might help along or that you may find extremely boring:
* Donepezil is a great way to cause more REM, other drugs that act in complex ways on the cholinergic system do too. Looking at trip report of these might be interesting. The big problem here is that most people taking these REM-increasing drugs do so during the day and have serious conditions (e.g. alzhimers)
* REM and memory consolidation are potentially inversely related (not sure if this is what you mean by "sleep renormalizes the hippocampus"), or rather, REM and short term memory consolidation may be, at least in that N2 and N3 seem to play a big role there and increase N2 and N3 seem to => short term memory consolidation. And I'd usually associate a longer REM with a longer N4.
* Pulse oximeters can kinda-maybe measure sleep stages well enough to be not noise and so can some comfortable 2-4 neuron EEGs. These will never be used in a study *but* trying to get feedback on your sleeping patterns and what helps might be a much better way than speculation here.
* (Maybe obviously) Short/Long sleep studies with depression may be hopelessly confounded by common drugs (caffeine, nicotine, alcohol, benzos) which affect sleep a lot, and which depressed people would be prone to take, and unless you've looked into the methodology closely it might be worth batting an eye over (though maybe you did and you think it controls for this properly => ignore me, but it seems like the kind of thing that's hard to deconfound for many reasons, one way or another you end up with weird samples)
So we've established in this post that depression is caused by sleep, particularly REM sleep. It's well known that REM sleep is the only kind of sleep where dreams occur. When dreaming, your brain generates some arbitrary thing-that-resembles-a-narrative-story, and this thing further reinforces the intuitive idea of being a self-controlled sapient character in a grand narrative of the universe. In Buddhist/ish thinking, the idea of being a self-controlled sapient character is a myth and being enlightened causes one to intuitively grok it as a myth. So chaining the logic together we should conclude that enlightenment is a cure to depression.
I'm kind of stumped on the dreaming phase of your daisy chain. I'm predisposed to think of dreaming as the opposite of narrative reinforcement -- more destabilizing of one's habitual reality.
Have you polled enlightened Zen monks on how much they dream?
No, I have not polled enlightened Zen monks (or any people with a strong spiritual practice, for that matter) on how much they dream, and I'm most uncertain about the relationship between dreaming and narrative reinforcement as well.
I've had both sever insomnia and MDD, though whether it ever qualified as 'severe' depression depends on where you put the cutoff I suppose. In my personal experience sever chronic sleep deprivation feels almost identical to depression, with the exception that it is much easier to get rid of (just catch up on sleep). Acute sleep deprivation such as is used in the attempts to treat depression, is significantly better than the depression itself if and only if enough caffeine/other stimulants are involved that I can maintain some level of actual brain function.
Yeah, I think people are underestimating the kind of sleep deprivation involved to shift the depressive state. It's not "oh I feel really tired, I'd love a nap" levels, it's "Mr. Brain has left the building, I am starting to taste colours" levels.
I mentioned this in another comment above, but in college I found for a period that reaching a certain level of sleep deprivation would relieve me of a lot of my anxious and depressive thoughts. Of course this was sleep deprived to the point of functional impairment (body pain + inability of deep thought), which grew worse the longer I kept this level of sleep deprivation.
The short of it is I would prefer being functional and sleep-deprived to being functional and depressed, but I would prefer being functional and depressed to non-functional.
Never been formally diagnosed so take this with the appropriate scepticism, but that's a really tough question.
You can kind of get used to a functioning level of depression day-to-day, and it's only the really bad times that make it really bad. Being sleep-deprived to the manic state certainly lifts you out of the normal depressed state, but mania is its own problem. With me, walking into walls and bursting into inappropriate fits of giggling wouldn't be too good at work.
So I guess - for myself - I'd take the daily functional misery over the sleep-deprivation, but for the really bad times I'd take the mania instead.
Depression and anxiety: I find that if I am consistently getting less sleep (e.g an hour less a night for a week) I have increasing difficulty with basic mental tasks. Also the physical side effects of sleep depirivation of general bodily pains and getting sick more easily. Any uptick in mood is relatively minor and doesn't help much if I'm struggling at general life tasks
Its been years since I did the staying up all night thing, because it would mess up my sleep pattern for weeks afterwards, so wasn't really worth it
As others have already commented, sleep-deprived to the point of 'mania' is generally better than (deep/severe) depression. But, as someone that suffered from chronic sleep deprivation (that was 'environmental', e.g. due to a noisy sleep environment), I'd take everything short of suicidal depression than that. A night of no sleep, or a week or two of too-little sleep, can help depression, but anything longer or more severe than that can lead to psychosis, which can be much (much) more unpleasant.
Stupid thought on REM vs non-REM section: depressed people actually have MORE synapses, not less. Depression is a brain wave "brownout": barely enough to run everything at minimum capacity assuming nothing else taxes the system. As more synapses form throughout the day, the brain has to upgrade to full "blackouts" in various regions to keep the brain from permanently damaging itself, which frees brain waves to run everything else better, which is why they seem to get better as the day goes on, and why sleep dep seems to make things even better: more blackouts free up more brain waves for the rest of the brain. They get just enough non-REM sleep to normalize back to brownout levels, which starts the cycle over again. I would expect this to be falsified if the entire brain gets reduced function instead of just isolated regions during depression, and I would expect brain frying drugs like marijuana to help. I'm probably totally wrong though and welcome correction.
Pushing back on the "number of neurons doesn't change" claim, neurogenesis (new neurons being born and integrating into the circuit) is not only a thing (and despite some claims pretty likely to be happening in adult humans for a limited number of cell types) but is also proposed by some as the main mechanism for the positive effects of SSRIs, exercise, enrichment, etc on mood, since all of these increase both neurogenesis and mood.
If seizures help treat depression, then why are people with epilepsy more likely to experience depression? Or do electroconvulsive seizures work differently than epileptic seizures?
In my view to consider sleep deprivation as a valid approach is nothing short of insane. Reminds me of lobotomy. As a side note, the underlying specific assumptions of a computer-like functioning of the brain seem to me a strange caricature of current knowledge. I guess I hadn’t been connected to the community for quite long and forgot the strange belief system at work..
If rem sleep clears out the short-term memory and a having full short-term memory bank is linked to the reward system that would make sense from a more intense morning depression and lesser in the evening standpoint. Would then refilling of the short-term memory banks upon waking by reading something particularly interesting/enlightening (such as this blog) create a large number of new memories right off the bat and kick-start the day with a more elevated mood due to the well topped up short-term memory? Are there more effective ways of filling the short-term memory than reading, such as cramming Trivial Pursuit cards or listening to an audiobook on double speed (doing that feels like an information download straight to the brain)?
So the overall result is to increase "equality" of synapse strengths throughout the affected regions? This sounds kind of like destroying information, but maybe if the information is the pattern of connections making you depressed that's good?
From an information theory perspective, increasing equality doesn't necessarily reduce information. Suppose the brain had 1 really really strong synapse. That isn't much information. It isn't hard to point out which synapse is the strong one. On the other hand, if all the synapses were equally weighted, that carries no info. (But might make the info about where the synapses are easier to access.) Much more info can be stored by giving half the synapses a weight of 2, and the other half no weight. The distribution of synapse weights that maximises info (given fixed total synapse strength) will conceptually resemble the distribution of kinetic energy in an ideal gas.
The actual neuron weights will follow a geometric distribution (in this simplified model, where we are maximizing info with a bounded total weight and weight is just the count of some molecule), with a parameter that depends on the total weight bound.
TL;DR. A long tail of really strong synapses doesn't hold much info. A more even spread of strength can be more informative.
When I was in college, I was giddy-drunk if I went without sleep for a while, and that was fun. (Although I didn't drink in college, so I can't really compare.) I think my inhibitions just went way down and I'm normally too cautious.
Looks like they understand depression about as well as digestion. I can tell you for a fact that living with an optimist who loves you but won't put up with the attitude after a point can cure depressive tendencies over time.
>Is it fair to say that "decreased synaptic strength" is sort of the same thing as "abnormally low confidence on predictions"?
1. Consider that what may be involved is something like a "ready to hand" instrumental confidence in the effects of the {depressed, manic} person's actions; we can try to gloss this as corresponding to the epistemic confidence they have in their predictions, but that abstraction may be misleading in a lot of cases. (Famously it's hard to translate the ability to tie your shoes or execute a tennis serve into "predictions" or "decisions", and a lot of quotidian executive-function tasks are likely like that.)
2. What may be going on is that the depressed person's brain is *too good* at making the synaptic connections. If this manifests over the course of months, we call it a bipolar disorder. If their brain is somehow wise to what's going on and adjusts daily (nightly), then they wake up despondent/hopeless/directionless, go to bed slightly manic, and are at their most average functioning mid-afternoon.
3. This meshes with a lot of other pathways/covarying cognitive deficits. This kind of depression is connected to anxious behaviors like compulsive procrastination; that would make perfect sense if ingrained compulsive habits like e.g. checking blogs (and writing blog comments?) provide extremely predictable feedback. (This is something I think makes more sense on a "ready to hand" model than a literal "epistemic confidence" model; it's not clear why doing something just to be doing it would result from reliable predictions, since anxious procrastinators can usually predict when prodded that nothing valuable will result and they will regret procrastinating.)
4. It explains some stylized facts like: (a) why melancholy tends to attack people in relatively independent careers (they have more options and are vulnerable to decision fatigue early in the morning, and have fewer forced actions to get them through their low-functioning part of the day); (b) why they tend to have self-perpetuating circadian rhythm problems at night (they are most overconfident in the value of doing one more thing when they need to go to sleep, and have very firm convictions about tomorrow's agenda that they meticulously plan out when they're lying in bed); (c) why some people are convinced lifting weights helped their depression (very simple practical cause-and-effect relations, usually early in the morning); (d) why mastering some new hobby with quantifiable success/failure/improvement metrics often brings people out of a funk for a while.
I myself was in a melancholic funk a long time ago, and it stopped at the same time I started a ketogenic intermittent fasting diet that involved going to bed hungry and not eating until I finished my morning workout. At the time I thought that my rising spirits gave me the willpower to follow the diet, but now I wonder if it wasn't the opposite; starving my hyper-activated synapses in the evening forced me to bed and decreased the amount of overnight synapse-correction I needed. — Meanwhile I also wonder if the problem didn't originally arise because for a long time I tried to save time and diet by skipping breakfast, habituating my brain to absence of sugar in the morning. (Consuming my morning caffeine with artificial sugar instead of real sugar probably didn't help.) — That said, I wasn't at my most productive while practicing intermittent fasting; my most productive years in recent memory were again a period where I woke up in a fog, got nearly everything done 1pm-5pm, sometimes felt like trying to get a lot done at midnight, and sometimes had sleepless nights. So it's possible the overall pattern of strengthening synaptic connections over the course of the day isn't purely dysfunctional or diseased, even if it carries risks.
As someone with lifelong depression, I can attest that the sense of being "fully activated" only late at night is very real. I do my best creative work between about 10pm and 2am. I used to intentionally plan all-nighters when I had difficult school work or papers to complete; I eventually just realized that if I wanted 100% productivity, I had to do it late at night.
At the ends of semesters, I'd sometimes take a 36 or 48-hour stretch of final paper writing during which my productivity would reach *extreme* heights. For example, producing multiple graduate-level papers which received high grades in just a few days - something I *cannot* do normally. Toward the end of these periods, I'd experience a sensation I'd describe as feeling "too awake" - extremely alert, quick thinking, rapid ideation, borderline manic. If you've ever taken ADHD medications in a somewhat too high dose, it was a very similar feeling.
So, this discussion seems extremely accurate to my personal experience.
" I'd experience a sensation I'd describe as feeling "too awake" - extremely alert, quick thinking, rapid ideation, borderline manic"
Yeah, I've had similar feelings the few times I've been very sleep-deprived (no or very poor sleep). That's why I'm a bit dubious about "keep the depressives awake and it will reset them" because 48 hours with no or little sleep will make anyone manic.
Also the fun sensations of "tons of energy, mind racing, and walking into walls, doors and anything in the vicinity because for some reason my legs won't work on straight lines anymore" after 36-48 hours wakefulness 😁
So, I used to do this in college as well. I only stopped because the "hangover" from doing this kind of thing ended up being more disruptive than the benefits I was getting.
I 100% can program better at 3 a.m. then I could at midnight which was, in turn, better than I could program during the previous afternoon. The commenting gets way worse, but the solutions are much better. This is useful to me, and I'd love to get to use it more often.
The problem is, the effects of doing this kind of thing just last forever. A 36 hour period of wakefulness has me feeling like crap for around 1-2 days. Anything over 48 hours give me cold-like symptoms for up to a week. Is there any way to get the benefits of staying awake without the aftereffect?
Is it possible to self-medicate depression by restricting sleep or reorganizing your life around it somehow? I'm thinking about shifting your work and important stuff to the evenings/nights and instead taking it easy in the mornings. I was reminded about how as a student I tended to gravitate towards doing more studying in the evenings (except for bar nights I suppose). It seemed to work then, and aside for a preference for having evenings off I don't see why it wouldn't work now as well.
This would seem to support a more complex view of the role of REM sleep in depression. SSRIs reduce REM sleep and fight depression; alcohol reduces REM sleep and worsens it.
Throwing another anecdotal evidence on the pile, I've always been a night owl, I find mornings difficult even though I've had to get up early for years, and on top of that I tend to have insomnia - either it takes me forever to fall asleep, or when I do fall asleep I keep waking up throughout the night.
And then when I get the opportunity to catch up on sleep, I can sleep for hours upon hours. I don't know how much linked that is with depression, but it certainly *feels* like it.
The times I have been sleep-deprived, I have ended up bouncing off the walls, but I don't think it had much of an effect on my depression; instead of feeling unmotivated and 'I hate myself', it was "Whee! I can finish all these tasks! *And* I still hate myself! Oh look all the straight lines are all bendy..."
What of other animals? Bats sleep much more than people, but have much simpler brains. Wouldn't you expect the much greater number of synapses to lead to some notable difference in sleep patterns or the types of chemicals released in sleep or something?
I was also thinking that dementia stems from a sleep disorder. It’s just that my mother who had something going on (neurologist could not define it other than dementia) would have moments of clarity then drift off to what seemed like she was describing a dream when she spoke. Meaning everything she was saying was disconnected like what happens in a dream.
What does this imply about people who need way less sleep than the average person?
Back in college the seniors would have a rotating shift of "night watchmen" who had to wake up and patrol the dorm a couple times a night to make sure nothing crazy was going on. When it was my turn I remember be constantly unnerved that the same 5 or 6 people out of a thousand and change were always up and about at 3 a.m.
Could it be that the "normalization waves" of these folks are abnormally strong and accomplish the task in just a couple hours? Or maybe they are susceptible to depression effects and have just evolved a strategy of exposing themselves less to sleep to maintain better mental health?
Glymphatics anyone? Searched for this in the comments along with lymphatics and aquaporin and go not hits.
Recently(ish) the extracellular mass transport of fluid has been shown to double during sleep. This is thought to be protective for Alzheimer's disease for example. It is also highly dependent on aquaporin 4 expression.
I note that early on Scott writes "As if depression is caused by some injury during sleep". The opposite could be true.
Hypothesis: some semi-soluble compound in the extracellular space is beneficial to depression but sleeping allows it to be washed away. During the day it reaccumulated.
So this coincides with a weird issue I'm investigating in my own brain.
For background, I have atypical depression (diagnosed, used to be medicated, mostly in remission for now) and probably ADHD (diagnosed as something ADHD-adjacent as child when that was not a category in my country, unmedicated). Mornings are universally the worst part of the day, both in terms of mood and sleep latency. Coffee helps a little bit but is mostly drank for pleasure. Brain fog is present during the entire day, but I manage.
The thing I discovered by accident is that drinking coffee in the evening gives me superpowers. I never liked to work in the evening, instead reserving it for chilling out, but with a sufficient burst of caffeine I get an incredible amount of stuff done, concentrating effortlessly while in a hypomanic state. Distractions that normally completely destroy my productivity are no problem - I can easily sustain conversation via IMs and not lose track of work, for example. This effect works _only_ in the evening and late afternoon.
This makes absolutely no sense under the standard dopaminergic (no relation to circadian rhythm) / adenosine antagonist (related to circadian rhythm, but it just makes you feel non-sleepy) explanation of caffeine action. The circadian depression voodoo explanation is... tempting, to say the least.
"This sounds kind of like destroying information, but maybe if the information is the pattern of connections making you depressed that's good?"
If I've got a textbook in mostly nine-point type, but one random sentence on each page is in forty-point bold, I don't lose any information and I do gain more efficient access to information if I reformat those sentences to the same font as the rest.
If it's not random sentences but the most recent sentences (the stuff I experienced yesterday and was supposed to reformat down to standard-archival-format), that sounds like it might combine synergistically combine with some poorly-chosen trapped priors. If my recent experiences are of the "life sucks, there is no hope" variety, and those keep getting overweighted, then I won't be able to draw effectively on my previous experience of life not sucking.
In which case, excessive sleep alone won't always cause depression, but would aggravate other potential causes of depression and impede natural recovery.
What about a simple alternative explanation: patients are going through withdrawal in the morning, because they haven't taken their SSRIs or other meds in 8+ hours.
Huh! I've noticed that during depressive episodes, my dream content "lags" by a few days. For example, if I watch a movie, those characters will show up in my dreams three days later, where normally they'd show up the same night. Related, maybe?
I have an interesting perspective of this, having done night work for over 30 years. Not just 30 years but for much of it six nights a week. Anyone who has worked the third shift can relate to the unnatural sleeping patterns. My day off (Friday--today in fact) I spend up to enjoy life the way day-trippers do. I am just about at 24hrs and going strong. Now I would not want to take a long drive at the moment, but lets hope this has some clarity.
Regarding depression, my own personal experience has been that sleep, deep sleep, is much needed to ameliorate it. Now, there is merit in sleep deprivation. The longest run I've had is three days straight. I fainted eventually (at the breakfast table!) but here's the odd thing--on the way home, I felt like a million bucks. A few things played into this I believe. The weather was warm and sunny, I kept myself moving (stopping is the bane of the sleep deprived) and was relatively trouble free. A miserable, rainy day in the winter after my cat just died might have another outcome. Never-the-less, I still fainted. My body pulled the plug on my mind.
Over the years, I have noticed a curious pattern. While I can stay up for long periods of time, I find after deep sleep, my mind fires on all cylinders. Let me tell you about my old friend Frank. His father was in a rest home and I would visit with him from time to time. Frank would warn me his pop would need a good 30 minutes to get "his brain moving" as he described it. He said it was almost like the blood had to get to his brain and took that long. Slowly his pop would come out of the malaise he was in, know our names, engage in conversation. I said I often stay up all day Friday. Depression (when it hits me) is Saturday morning. Early. 5am maybe after a good 5 hour sleep perhaps. I could lie there turning over thoughts that somehow seem the have the better of me. So, I one morning thought of Frank's pop. Thought perhaps my own blood had to get flowing, and simply chose to let my mind worry later, at a time when my mind was better apt to take the trouble on. Just as I would not fancy a long drive now for safety issues, I applied the same to my mind in that state. Fast forward to Monday night now. I am driving in, bright and alert, same troubles and "I've got this', I can say to myself. All my mind needed was rest. I've become a big advocate of proper sleep to bring clarity and convinced lack of sleep causes shotty thinking. I don't deny the research here, to some degree have experienced it, but also the benefits of a rested mind to achieve the same. I've seen the depression "reset" Saturday morning but the exact opposite in direct proportion Monday night sleeping sound all day.
I've often felt like i get more tired and depressed towards the end of the day. I have a demanding job, two kids, and aging parents. Towards the end of the day, I often feel down about almost everything. That seems to be the opposite of what the article describes.
When i wake up, this generally isn't the case. I've often used the image of something like 'light' that illuminates the space of possibility nearby - both what's happening now, and what might happen. When I feel good, this light is bright, and when i don't feel good, it's as if the light is 'dim' - everything just seems more dismal and hopeless.
That might make sense of connection strength gets stronger over the course of the day, and strong connections require more energy to maintain. If the connection strengths all get downgraded evenly over the course of sleep, then it takes less energy for my brain to map out all the various connections between things in my immediate environment and, as a result, i can more easily perceive the subtle, positive things that make life seem better.
It often feels, when I'm tired or angry or sad, like my experience of the world reverts into a 'cartoon' state, with all of the unpleasant feelings highlighted. That might seem consistent with only the strongest connections getting enough energy to run?
If we follow this theory that (certain patterns of) increased synaptic connection treats depression, and evidently our activities over a day tend to produce such, do we know if there are particular activities that do so disproportionately? It might make sense to front-load those.
There are some candidates on how top-down theories like predictive processing and bottom-up theories like homeostatic disregulation can meet.
In predictive processing, there are two major candidates on how/where the downwards prediction and the upward sensory information might actually be implemented.
One is that different parts of the layered structure (https://en.wikipedia.org/wiki/Cerebral_cortex#Layers_of_neocortex) of the cortex are used. Details are complicated, but probably upwards stream mostly arrive in layer IV, while the comparison with prediction might be done in layer V, or possibly in layer II/III. We don't understand the details of this "canconical microcircuit" very well yet, but there is a lot of structure.
The other (complementary) candidate is that different neurotransmitter are involved. Upstream information uses mostly AMPA (which is a LOT faster and is responsible for the lion share of energy consumption), while predictions probably/perhaps involve a lot more NMDA receptors (slower and longer-lasting, strongly involved in plasticity and learning).
Homeostasis does not affect all these structures the same way. As far as I understand Tononi's theories, mostly the AMPA part is upregulated over day and downregulated at night. Also, different layers are affected differently.
Once you know such things, you can start to speculate. If AMPA is stronger in the evening, does that mean that predictions are relatively weakened compared to sensory information? I don't think it gives a coherent picture (yet), and I don't think that we are at a point where we can match top-down and bottom-up theories. Probably we won't be there in the next 10 years. But I don't think it is hopeless in the long run.
Maybe but I still out substantial credence on finding out the sleep result reflects affects to the endogenous opiate system/cycle. I'd love an experiment that measured levels of those endogenous opiates over the time period.
What about schizophrenia? Sleep deprivation often leads to psychosis and the negative symptoms are a lot like depression symptoms? What's the interaction there?
It's weird to me to hear that sleep deprivation can help with depression, because I'm pretty sure sleep deprivation makes me *more* depressed, at least in the long term. But possibly short-term sleep deprivation is helpful and long-term sleep deprivation is harmful?
During college, I did a research paper on synaptic scaling during NREM sleep, and ran into Tononi on a ludicrous number of articles. I started noticing his name popping up on all the interesting ones, so I just did a search by author. I don't know how the **** he does it, but his name is on all sorts of crazy papers that are clearly trying to become a groundbreaking paradigm shift in their own way. At this point I'm convinced that he has a network of contacts across the entire field that give him early-warning alerts on weird new papers so he can swoop in and coauthor them.
For what it's worth, btw, the consensus among researchers ca. 2018 was shifting towards Tononi's NREM model being less helpful than the other big model in the field. There were a couple of studies that looked really slam dunk until one of the big names on the other side pointed out that they were anesthetizing their mice with an agent that specifically interfered with the part of the brain they were looking at. I don't know what the fallout was from that or if Tononi's people recovered, but it was an oof moment for sure.
Huh that explains it for me, thanks! "It" being: I sometimes feel depressed early in the morning, like first 5-30 minutes after waking up, although I'm normally not and never been diagnosed. It happens without any apparent reason like sleep deprivation, hangover or shitty day ahead/behind. And it's not just low energy, I literally feel like I'm worthless, my life sucks, things I said and done yesterday are all failures etc, even though I remember it's not what I was thinking yesterday and know I'll probably feel differently in half an hour. I eventually assumed it's some sort of a brain glitch and just made a habit of not thinking of any such matters right after waking up, now I finally know what kind of glitch this (probably) is.
As someone who experiences depression, it personally feels more like “your emotional system only has half as much RAM as usual today, so you're not going to be able to trust anything you think”. I don’t feel sad per say, I just don’t feel good either. Whenever someone asks me what I would like to do/eat/see/whatever I say “I don’t know” because none of the options seem like they would be better than the others. I feel like maybe I would be better off if I were able to at least keep strong negative associations with things in mind, because then I could figure out what I want though process of elimination.
In case anyone still reads these comments, since I read this article when it first came out I've started viewing nights where I have to sleep less as medicinal. "Okay, I can only sleep 5 hours tonight. That's good! I've been sleeping too much and feeling sad. Maybe tomorrow I'll feel better. :)" It might be a placebo, but so far this mindset has made me 1. more of a morning person. 2. happier. :)
I have always wished I can offer myself for any well funded research into the connection between sleep and low grade chronic depression. I have always believed and told people that I feel there's a strong link between the two phenomena.
I sleep a lot so much so that I was nicknamed Jonah the sleeper in childhood. I once went to my university health center to complain about sleeping too much. I meet many of the criteria for Narcolepsy and all the criteria for hypersomnolence. I once took modafinil (which I first heard about on this blog) and it worked tremendously which made me stop for fear of dependence. But it didn't relieve my depression, only excessive sleepiness.
Much of what this article says about the complicated relationship between sleep and depression applies to me. I have long discovered that I experience a negative rebound from the kind of overactivity associated with having fun such that I almost always have a mood crash afterwards but I have tended (obviously wrongly) to interpret it in psychological terms: I feel guilty after having so much meaningless fun such that I feel the need to atone by returning to my true original self. Alas, if this article is anything to go by, it is just pure circadian chemistry!(?)
Relationship with ketamine treatment?
Find out at https://sci-hub.st/https://pharmrev.aspetjournals.org/content/72/2/439.abstract , then summarize it for me, please.
Sorry, that's blocked for me; I'll see if I can get to it on a different computer.
You could also just ask Google Scholar for one of the many possible fulltext links: https://scholar.google.com/scholar?cluster=4200585988599354109&hl=en&as_sdt=0,21
Sci-hub is your friend. The domain changes periodically, which is why I'm not linking it, but you can just give it the DOI of the article and voila, there's the PDF.
What I was hinting at was that this article is open access and it's more generally useful to start with GS and only move to SH when that fails...
Hey folks-if the initial link is blocked, there is no way for me to have enough info, like the DOI, to search for it. Gwern's link was helpful.
Yeah, I had that reaction, too, but chopping off the sci-hub part of the original link does seem to get you a working journal page:
https://pharmrev.aspetjournals.org/content/72/2/439.abstract
There's even a PDF button.
figure 3 A says: "During the development of depression, susceptible neuronal networks become gradually
hyper- (red) or hypoactive (blue) under predisposing environmental conditions, and they retain that state at the expense of optimal network
functionality. This may manifest as, for example, uncontrollable self-focused rumination, depressive thoughts, anhedonia, and cognitive dysfunctions.
Rapid-acting antidepressants (i.e., ketamine represented in the figure) have excitatory effects that are reflected in the activity of local circuits, leading
to increases in cortical synaptic strength and the re-emergence of global functional connectivity patterns. This breaks the recursive cycle of rumination
and allows for the immediate relief of depressive symptoms."
Thanks!
Typo thread
The parenthetical beginning "Giulio Tononi (equally famous" is never closed.
Another minor point: the common abbreviation for transcranial magnetic stimulation is TMS, not TCMS.
Rantamäki, not Rantämaki (sorry, very obvious error to a Finnish speaker)
Wow, would you please explain why? (ä is rare in the second syllable in Finnish?)
Firstly because Rantamäki means something whereas Rantämaki does not.
Ranta = beach, mäki = hill
Secondly, you never have a and ä in the same word (like rantä). unless it's a composite word.
Räntämaki would be a proper word, but sleet-lemur seems like an improbable surname compared to beachhill.
Great, thank you very much!
Not sure if a typo or not, but
"some people will start at the hardware level and work up, other people will at the software level and work down"
seems a bit like it's missing the word "start"
Rantamäki was spelled wrong in slightly amusing fashion.
Rantamäki translates to "beachhill", whereas rantämaki sounds like an experimental vegan dessert of partly melted snow wrapped in a nori sheet.
All makis are primates, that part is clear, but what is a rantä?
No, no, maki are the rolls you get at a Japanese sushi place.
My mistake, was sure maki was the same as in German: a lemur or macaco.
I think you’re right
Rantä doesn't mean anything (and breaks vowel harmony); however räntä means rain of wet snow.
Finnish would have a word for that, wouldn’t it?
So does English: sleet
On a note that is mostly unrelated to the OP, I unironically endorse anyone who wants to sell such an experimental dessert purely for the humor value.
(Having eaten both snow and seaweed, I think that in terms of taste only this would be mediocre.)
I think something like a slightly sweet cucumber-wasabi sorbet filling would be true to the concept, but also potentially delicious.
Firing powerful magnets at the brain sounds likely to cause cranial trauma. Maybe try magnetic fields first ;)
Not a typo, but the divide by 100 example of renormalization is wrong. Normalization implies that the vector is being made normal, for example having a "normal" magnitude. A better example would be dividing all the numbers (0.9+0.5+0.3), so that they add up to 100.
I'm not sure about how AI-ers use the idea of renormalization (a quick search turned up a paper titled "Neural Network Renormalization Group", which putatively describes neural networks using a renormalization-type idea from physics, and a number of other completely unenlightening papers/writings), but presumably I think it would mean just dividing stuff by an appropriate constant to fix its scale. Also sqrt(0.9^2 + 0.5^2 + 0.3)^2 = 1.073 ~ 1, so even if we are uncharitably assuming the exact mathematical definition of renormalization Scott Alexander's example is not too far off the mark.
Not totally sure what Scott actually meant by this, but the two almost ubiquitous practices in statistical modeling in general (not just deep learning or layered models) are 1) clamping data values to a small continuous range like [-1, 1] and scaling everything based on distribution parameters of the known inputs, and 2) penalizing complex models via something like L1/L2 shrinkage (the minimum description length principle from a Bayesian perspective). The former I would call "normalization" (typically you're assuming a normal distribution, which means you can just do x' = (max(xs) - x) / (max(xs) - min(xs)), maybe after removing some obvious outliers, and you've just clamped) and the latter I would call "regularization."
I don't know much about physics, but don't *think* this is the same thing as renormalization in that field, except in the sense that both involve making assumptions about data and manipulating numbers based on that. This seems like quite a bit more of an exact science, so to speak, in physics, just because the data generating processes are much better understood.
Although, for what it's worth, when L1 regularization first came to linear regression analysis in the 80s (L2 had already long been in use and was known as "ridge regression"), the idea also came from physics. Normalization in the sense I'm talking about is different. There is no theoretical basis from probability guiding why you'd do it. You're doing it because of numerical stability of the training process, to avoid having your gradient estimates blow up because of really big input values. Regularization, on the other hand, involves making theory assumptions about the nature of the maximum a posteriori model.
In physics, normalization so that the probability of finding an electron over the universe integrates to 1 comes to mind...
Probably not important, but the "eg" in "see eg Scott Aaronson's discussion here" should be "e.g.".
Is it silly to notice the superficial similarity between EMDR and REM (both involve moving the eyes) and effects on psychiatric conditions? Might the evidence for the mechanism of action for one be relevant to the other?
Eye movements are controlled by the frontal eye field and are typically the only part of the body unaffected by atonia during REM sleep, thus REM occurs as we dream. There are two hypotheses of the cause of EMDR efficacy in treating trauma. The first is that eye movements or a different bilateral stimuli such as tapping or vibrations (yes, EMDR does not always involve eye movement!) are in some way responsible for “activating both hemispheres” or some equally hand waves and implausible mechanism. It might be right but it doesn’t make sense to me. The second hypothesis is that the eye movements or bilateral stimuli are purely placebo and that what really works is imaginal exposure. Having successfully treated PTSD with trauma focussed CBT alone, I favour the second hypothesis. I’d therefore conclude that the eye movement is a coincidence but would happily change my mind if a plausible eye movement in EMDR theory was put forward
There's also the theory that following the alternating sides stimuli in EMDR continuously triggers an alarm reaction.
The alarm reaction is hard wired and partially blocks the remembered trauma, so that it is not able to overwhelm (and thus not desensitize, or even re-traumatize) the patient.
Anecdotally (both from applying or being treated with EMDR), jagged irregular zigzagging motions of the therapist's hand waves require a stronger focus in the patient, are triggering a stronger alarm reaction (as felt in one's alertness), and thus have a stronger shielding against flooding effect.
Also the classic hypnotist's pendulum - "you are getting very sleepy...". There's a lot of stuff like this and I don't have a good understanding of any of it.
In a recent post you hypothesized that "paying special attention to the sensory input channel" was important in EMDR. Is there a just-so story we can apply to REM that's consistent?
https://resources.hypnotechs.com/spiegel-test
The Spiegel Eye Roll Test for Hypnotizability:
"The ability to look upward on signal while closing the eyelids (Eye Roll Sign) correlates highly (73.9%) with hypnotic trance capacity as measured by the Hypnotic Induction Profile in 2000 consecutive psychotherapy cases. In practical clinical terms this implies that in about five seconds the Eye Roll (ER) sign predicts hypnotizability in three out of four cases. Further, the higher the roll (0–4 scale), the higher is the trance capacity. This offers a quick, subtle, clinical way to ascertain whether or not hypnosis can have a probable role as an adjunct in the various psychotherapeutic strategies."
Have caution. In looking up just what the "Eye Roll Sign" was to better understand that first sentence, a top Google result was a 1982 paper whose abstract claims that the purported correlation is far weaker than advertised, to the point of being clinically useless: https://jamanetwork.com/journals/jamapsychiatry/article-abstract/492859
I just tried it and I found that I can kind-of do it. Is this 🦋 depressability*?
*For the confused: (putatively) passing Spiegel Eye Roll Test -> being able to be hypnotized -> being able to do EMDR -> lots of REM sleep -> depression.
It seems like I can do this, but I tried very hard to get hypnotized back when I was an impressionable teenager and got really into Edward Cayce and was convinced human clairvoyance was both real and something I could tap into if I could just manage to achieve a higher plain. It never worked. I was never able to be hypnotized.
Damn it, no edit button. That is supposed to say "Edgar" Cayce.
DISCLAIMER: please see Majromax's response. I agree with it. And should have included in my initial comment that I wasn't endorsing the information as accurate, but just meant to provide another data point for eye movement being a thing that the hypnotism community cares about, and a slightly more concrete example than "yeah they do the thing with a pendulum".
As a data point, I am sometimes depressed, but always feel least shitty in the morning, where I have the most energy and mental clarity, and feel most shitty at night. This seems to be one of those axes along which there's your pretty typical spread of variation; if depressives seem to disproportionately be more night owl than morning lark, that *could* be due to the effects speculated here, or it could just be an "everything correlates with everything else" thing.
On this blog I've heard a bunch of times about how sleep deprivation can help (temporarily) with depression, but out in the normieverse I only ever hear about how sleep hygeine is good for mental health. Granted that there are other reasons why "permanently sleep deprive yourself" (which depression might do anyway lololol) isn't a good idea, is this just the broader culture *assuming* that a generally healthy thing (consistent, full sleep) is good in every specific, or is it still best, including for depression? (It's possible this was answered somewhere in the technical details that I lack the brains and background to follow.)
To me, 'sleep hygiene' has always meant 'stable circadian rhythm', which I suspect is probably more effective than mere sleep deprivation in the long-term treatment of depression.
I will second this experience -- when I went through a several-month period of severe depression several years ago, I always felt relatively fine in the morning (just anticipatory), horrible by midday, and deathly by evening. My desire for sleep (general sleepiness throughout the day also surged), and I started sleeping far more: normally I sleep 5-7 hours if I do not set an alarm, but the amount I sleep surged to 11-15 hours without an alarm, and budgeting ~8-9 hours per weeknight (which was the most I could realistically give myself) seemed terribly insufficient.
After about four months of this it pretty much went away after a psychedelic trip.
Most of the time I have a similar pattern as you, feel crappy in the morning, feel crappy at night, feel less crappy in-between. This is actually the reason why I don't get enough sleep, even if my mid-day self says "I need to go to bed at 10 PM", my night-time self sometimes says "Tomorrow is hopeless so let's stay up till 3 AM to keep away the pain."
But I did have some interesting experiments in college with sleep deprivation. Part of this may have been that I was more manic in my college years (my illness is probably some sort of type 2 bipolar, although the manic side has become less prominent over the years, and this is coupled with some degree of an anxiety disorder), but I found if I reached a certain level of tireness I was too tired to think.
This was amazing for me, suddenly I could just tell myself do this and I would do it without any negative back-talk or anxiety wrestling. This happened reliably enough that I tried to maintain that level of tireness for one or two weeks. Of course the downsides showed me that this was unsustainable. My body felt like it was falling apart and because the whole point was I was too tired to think, I had trouble thinking deeply about things which began to affect my ability to learn new concepts. Eventually I had to catch up on my sleep.
I don't think what I'm talking about necessarily has anything to do with what Scott was writing about, but that experience has made me interested in the relationship between mental illness and sleep.
As in my other comment, I'm wondering whether the significant fraction of people with sleep issues who have depressive symptoms might be contributing to this axis. I.e. a lot of people with depression may feel bad in the morning because a lot of people with depression (>20%) actually have a sleep issue like sleep apnea that causes both feeling-bad-in-the-morning and depression.
This seems like the kind of thing people would have thought of, but then again, it seems like depression patients should be tested for sleep issues (if not right away, then at least after failing a typical antidepressant + therapy attempt), and that's not widespread.
I used to have depression-like-but-clearly-not-typical-depression issues that I eventually discovered grew out of a combination of sleep apnea and dust mite allergy. Up to a threshold of exhaustion, I'd definitely feel better the longer it had been since I'd slept, and there was probably an element of feeling better the longer I'd been away from my mattress/pillow. Having controlled both of those problems, I no longer have any depressive-like symptoms.
This has been my experience with people who suffer from depression as well - people who are depressed often feel worst when they are tired. My thought was always that tiredness led to them being less able to push back against negative mental patterns, and that being tired made standard mental patterns more common/prevalent as the body tries to shortcut as much as possible on thinking.
Honestly, "depression" probably isn't actually one disease at all, but a symptom of a number of different diseases, which is why people with "depression" respond so differently to different treatments - some of them have one condition, some another. Some people have "anxiety depression", other people have "bipolar depression", some people have "sleep depression", ect.
The fact that a lot of people with depression also have problems sleeping makes me very suspicious of the idea that sleep deprivation helps depression, as people with disordered sleep - and who thus end up tired more often - end up *more* depressed, not less, on average.
My experience with depression and sleep deficit has been it's worse at least in part due to...not pain, exactly, but the discomfort that comes with tiredness. It's hard to feel good, or even okay, when you've got a headache and your eyes hurt and you've got an ache in your muscles because you haven't gotten enough rest and your chest feels weird because you've drunk too much caffeine.
I feel crappy immediately after waking up (specifically before I get up and start having inputs, leaving me with just brooding), crappy at night (despite the inputs), better in-between.
I recognize this far more than the example in the article. I must ask, how fatigued were you? How anxious? Both of these can ramp up during the day by fairly obvious means, and lead to depression through unsophisticated channels.
If I understand this right, maybe the better sleep hygiene thing recommended for depression needs to be amended with restricting the amount of sleep you get, if you tend to oversleep like I do.
Would it make sense to "jumpstart" synapse density first thing every morning by reading/learning? Perhaps that habit would help keep depression at bay.
Doing anything interesting/active decreases depression - this is "behavioral activation". I don't know whether this phenomenon is involved in that.
As someone who often feels depressed in the morning, I have noticed that if I do something especially positive in the morning, it immensely helps me the rest of the day. The difficult thing is that when I wake up I feel like nothing is going to work, nothing is going to help, I don't even deserve to be happy. So even if I have a plan of, "okay, I'm going to take a 30 minute walk to start out the day", when I wake up in the morning I throw the plan away and then I get angry at myself for throwing the plan away which leaves me sometimes more depressed than if I did not have a plan in the first place.
But hope springs eternal, I do want to get into a pattern of doing something positive when I wake up, I still think it's a good idea and I still plan on trying... but it's hard.
My wife got a lot out of Miracle Morning by Hal Elrod. The Miracle Morning: The Not-So-Obvious Secret Guaranteed to Transform Your Life (Before 8AM) https://www.amazon.com/dp/0979019710/ref=cm_sw_r_cp_api_glt_fabc_2FQ3F44EVP3JA7FRA6RQ
What's the gist?
You wake up early and do several “savers”. “Savers” are different tasks that you perform like visualization, meditation/prayer, reading and journaling. I think there are more, but that’s the gist.
Have you tried a "sunrise" alarm clock? I found that helped significantly with morning malaise for me--the gradual wake-up takes a lot of the sting out of it. I made mine out of "smart" lightbulbs, just set a program for them to start gradually getting brighter and bluer over an hour every morning.
What about an alarm clock with a screen that shows cute kittens (or whatever makes one happy to see)?
Better yet, sleep with actual cats and they will gladly wake you up in the morning when they decide it's breakfast time. No need for a clock at all.
Heh, maybe that's part of the effectiveness of morning gratitude journaling.
Huh, can explain why reddit/twitter feels so attractive when I wake up
I wanted to explore this same idea. I notice I'm in a much better mood on a workday vs a weekend or holiday. I've noticed this for years and wondered if I just felt more distracted and preoccupied at work and didn't pause to think about mood during the day. This gives a different explanation. Further, I've noticed I like to keep my laptop by my bedside and start my day working on spreadsheet activities. I've recently come to think of it digital gardening. And again, this article gives a new explanation as to why this feels satisfying to do.
I didn't look too long but I'm wondering if to what extent they have tried extended sleep deprivation as a treatment. The link you provided seems focused on these 36 wake periods in the short term, not the long term. As in, rather than just see how symptoms are reduced over the course of one or five cycles of 36 hour wake periods or whatever number they've chosen, how are they reduced if this sleep deprivation is done as a consistent lifestyle? Obviously this would pose a problem with living your normal life (family or work obligations, etc.), but if you have a severe case of depression those may not be functioning well anyway and you might as well give this a try if a researcher is offering it.
I know the depression seems to come back as soon as they fall asleep again, but perhaps with enough persistence this effect could be reduced through some sort of positive feedback loop, knowing that at the end of the day you'll feel remarkably better and you only spend maybe half your time feeling like crap instead of all the time.
There are a lot of experiments with trying to find better ways to make this last long-term. The most effective non-pharmaceutical one seems to be a week of on-and-off sleep deprivation culminating in your circadian rhythm permanently shifting to a different bedtime. I'll probably have a Lorien Psych post up with the specifics at some point.
I expect in the process of coming up with that they tried something like what you suggest, but I'm not sure.
I always ask depressed patients about sleep. One strange thing in DSM and ICD criteria for major depressive disorder is the hypersomnia or hyposomnia. Too much sleep or not enough? Aren’t those completely different symptoms?
I like Tononi’s synaptic normalisation hypothesis and there seems to be good evidence for it. There’s also evidence that a lot of metabolic waste products and extra cellular debris are cleared from the brain via the ventricular system during sleep, so it’s possible there could be more than one purpose for the change in activity.
I seem to recall that the build up of substance P during sleep was proposed as being connected to the depressogenic nature of sleep. Conversely, the build up of adenosine during the day, to make us sleepy, was proposed as an antidepressive effect (I can’t remember how many years ago I read this), but this seems implausible as we’d therefore expect caffeine as an adenosine antagonist to worsen mood.
The more I learn about sleep, the weirder and more fascinating the links with mental health become. I have a patient who was diagnosed with a non-epileptic attack disorder by neurology, but who is describing many symptoms of narcolepsy with cataplexy, enough that we have referred to a sleep specialist who can do sleep and awake EEG and maybe a lumbar puncture to confirm hypocretin levels. The patient is also reporting frequent nightmares relating to past abuse and trauma, and I am wondering if their hypocretin levels are normal that perhaps psychotherapy for the past abuse might alleviate the nightmares, help the patient to sleep normally, and correct the sleep paralysis and possible cataplexy they are experiencing.
I’ve had patients with PTSD who were experiencing recurrent trauma related nightmares in *every sleep cycle* awaking about once every 90 minutes to 2 hours. Simple trauma focussed CBT fixed this, surprising even me. I’ve heard other therapists and patients describing massive improvements in sleep post successful PTSD psychotherapy.
On the seasonal affective disorder link, I note that melatonin is synthesised from serotonin, but have not been able to find any anatomist to tell me if the melatonin in the pineal gland is synthesised from the supply of serotonin manufactured in the serotonergic neurons of the raphe nucleus or not, so this may be a red herring. Still, we might except a hypersomnic depression to more closely resemble an SAD presentation and a hyposomnic depression. Of course you also know that mixed depression and anxiety or atypical depression is often noted to occur more frequently than depression alone.
It certainly is a huge puzzle
I have narcolepsy with cataplexy and it went undiagnosed for years, even after I went to a sleep specialist. In their defense, going over their notes much later I noticed they had scrawled in the margins that I might have narcolepsy.
For years I had no idea what was wrong with me but I knew *something* was. I'm in treatment now and doing so much better, life was getting borderline unlivable before.
I’m really pleased that your narcolepsy was diagnosed and that treatment is helping. I am unsure if my patient has actual narcolepsy or if the sleep paralysis is being caused by exhaustion due to impaired sleep in some way, hence the referral to the specialist. I will be very glad if the specialist can figure it out, as I guess you were.
Curious...what were the pre-diagnosis symptoms and what is the treatment that is working for you? I have frequent sleep paralysis and fall into immediate action-movie style vivid dreams upon sleep as well.
I don't have lots of sleep paralysis (though I have had it on occasion, not enough to be an impairment). I do have the hypnogogic hallucinations now and then. Really the only bad narcolepsy symptom I had/have was/is cataplexy.
The main reason my diagnosis was complicated I think was because I had been previously (correctly) diagnosed with Tourette's syndrome (and I have basically all the symptoms there, including the infamous coprolalia). Having one movement/behavior disorder on the charts caused pretty much everything to be interpreted through that lens, I feel.
In college I started to have really classic textbook cataplexy symptoms -- heightened emotional states or sudden noises would cause me to just drop limp and be unable to move for several minutes, while remaining fully conscious. I'm now treated with Xyrem, which works really great, but the government in their infinite wisdom decided that only one company should be allowed to manufacture it and that means they can and do charge monopoly prices (it's all an insurance racket though as if you can't pay they'll charge you a more normal rate).
Thanks. All the hoops one needs to jump through for a diagnosis...referrals, failure with other treatment, sleep study, etc. has never seemed worthwhile to me, but now that Xyrem is available that potentially changes the equation. Frankly, I'm surprised they managed to get it approved, even with all the special FDA rules around distribution, what with the hysteria surrounding GHB.
Xyrem, at least for me, has been magical. For my tourette's I was put through the typical pharmaceutical roulette wheel, trying a dozen different drugs, many of them off-label, trying to find something that would work, and all they did was make me gain weight, feel incredibly drowsy, depressed, and suicidal. I didn't have high hopes for pharmaceutical treatments for my Narcolepsy, but then -- whiz bang, here's this miracle drug with basically no side effects that nearly completely eliminates your symptoms. I really couldn't believe it. Not sure how typical the results are.
Xyrem has been pretty miraculous for me, too, even though I'm a decidedly noncentral narcolepsy patient (to the extent that several specialists have questioned whether I actually have narcolepsy.
There is also some suggestive evidence (https://pubmed.ncbi.nlm.nih.gov/11696070/) linking natural short sleepers to a kind of mild anti-depressed/hypomanic phenotype.
Which direction does causality go here? Maybe both, as a positive feedback loop that is limited by another factor?
Possibly related: Narcoleptics supposedly enter REM state almost immediately upon falling asleep (talking like 5 minutes). I'm Narcoleptic and it definitely matches my personal experience in having wildly vivid and epic dreams even if I just doze off for 10 minutes. Does that have any further implications for these findings?
>I think this is the position that Rantämaki and Kohtala are taking in their Encoding...
>Is this Rantämaki and Kohtala's theory?
You got the name wrong here. It's Rantamäki.
Ṩöṙṙÿ
No problem, Scot Alexxander.
On the abnormally low confidence on predictions / overly strong or trapped priors theory of depression, it might not be as much a feature of the whole brain as of particular areas involved in reward, reward prediction, and reward prediction errors that are causally involved. Although the whole brain as a neural network could be using Bayesian prediction to predict inputs as per the Bayesian brain hypothesis, when it comes to depression (and perhaps anxiety) it could be regions such as the orbitofrontal cortex, amygdala, and the habenula that are affected in reward, reward prediction and reward prediction error.
I'm nervous about asserting that because depressive symptoms seem to affect so many areas - eg psychomotor retardation, appetite, cognition, autonomic regulation, etc.
I recently reread your posts on depression as networks, guyenet on motivation and the chamber of guf and I think they are all relevant to the topic of depression (but I'm confused about the sleep part). Also, this comment will be way too long for the top layer, so I'll add it here for easy condensation.
epistemic status: I'm an engineer and I think a lot about feedback loops and modeling, but I'm completely ignorant of most psychology and psychiatry. I'm currently thinking about this from a predictive-processing perspective, but this is an ongoing experiment and I'm not sure myself where this comment will lead me.
So let's try a modelling approach.
For sake of the argument, let's assume we're building a robot. It's supposed to be autonomously deployed to unknown worlds (minecraft, other planets, household robot for humans, whatever) where it has to find resources on its own and actions will have a cost (it needs a positive balance for energy intake, oil, ...); the environment will have periodic cycles (day-night; seasons). And it will not only be one of the robots but many, so they could cooperate or compete for resources, but there won't be any central coordination structure.
Now, let's think about what kind of structure we could give our robots that'll maximize their chances of thriving in the unknown environment. It would need some general possibility for building an internal map of the world around it, so let's implement to a predictive processing structure that generally tries to predict actions and minimize the free energy of it's internal model. Probably balancing various needs would be OK and we could do a random selection of task priorities comparable to the mechanisms described in the chamber of guf / guyenet on motivation posts.
Now: how active should our robot be?
The obvious answer is: We don't know! It's probably quite clear what kind of internal structures it'll need (vision, motorics, a possibility to harvest and use ressources, reproduce, cooperate / coordinate / compete with others), but without knowing exactly what kind of world it'll be deployed in (how many easy ressources vs. how many traps are there? how cooperative will be the surrounding agents?), there's nothing we have in the predictive processing toolbox to choose a straight-forward algorithm.
So let's make the best we can out of it:
- do a genetic algorithm that randomly sets activity thesholds in various robots: If it works well, there should be a way to pass this level on to more robots.
- do a medium/short-term feedback loop based on rewards: in a high-success-rate scenario, increase the activity level; in a negative-feedback-environment, decrease the activity level. If possible, tie this to obvious predictors (e.g. increase activity levels at warm temperatures if these are predictive of a high-reward scenario)
- include a slowly moving feedback mechanism that slowly changes the threshold over time; As we cannot tell how high/low the threshold should be, I assume we'll set an "expected number of activities per day". Each day we do more than intended, slightly raise the threshold to avoid more activity than the robot can handle (usually very high activity is very very costly). Set this to a very-sowly-moving timescale so this will build a homeostasis over months, but not override the medium/short-term feedback loop described above.
There are a few obvious ways something like depression could rise in our robots:
a) the robot just has bad luck and recieved an inherently very high activity threshold in our genetic lottery.
b) the robot's internal model is not calibrated very well and for whatever reason, predictions don't work as intended. Model calibration within a hierarchical model is really hard, so maybe there's something like to loose priors that make prediction of the right task at hand not work out as required by the current environment.
c) something really bad happened and pushed his feedback system out of the limits. Hopefully, it'll rearrange itself after some time, so unless the bad episode destroys his complete internal model structure of the world, it'll probably pull itself together sooner or later. (note that this could be a "big traumatic event", but also just an unintended prediction error, depending on this individual robot's calibration for how long-term it's planning strategy should be and how hard it's trying to maximise "always be right with your predictions" by having a really large feedback punishment for unexpected outcomes)
d) unintended feedback loops: This is an interesting one as there are various feedback mechanisms working against each other, so let's explore it a little bit:
Suppose something good happens, so the "be more active" feedback mechanism kicks in... so the robot will be more active, possibly getting more positive feedback, and so on. At some point, the slow compensation mechanism decides to raise the bar of "level of potential required for action to be taken". As a lot of good feedback is streaming in, this will override a slightly raised activation threshold, until at some point the threshold is raised really high AND something unexpected or bad happens. Now the stream of good results is broken and the activation threshold is high, so the robot will basically sit there and do nothing. After a long-enough time of being inactive, the activation threshold will finally be low enough that the robot takes some actions... leading to the next stream of positive events...
What does this mean for depression in humans?
a) and b) are interesting because they are "genetically inherited". I'm not even sure they are the same cause, but from the outside, it'll look the same, as the robot will have a lot of reluctance to choose action. I'm not sure if they would feel the same to the robot (if you could warp yourself into two robot's counsciousness, that is).
c) could be a really bad outside event (like losing a lost one) or a very traumatic experience (like being in a bad accident and losing a limb). In this case, the internal model would need a lot of recalibration (to accomodate both for the prediction error of not forseeing the traumatic event and the changed status of the world, e.g. no longer using your arm if it was damaged in the accident or having the lost person as a core part of your social structure). Again, I'm assuming that how bad this hits the robot partially depends on it's internal thresholds for sensitivity, but also of how robust the internal structure is right now: is the recalibration a feasible task or almost impossible as the event ripped out an integral part?
d) would look a lot like manic depression, cycling between very active and totally passive phases. It would also look a lot like known feedback loops, e.g. the lynx and hare predator-prey cycles.
From the outside, this would all look very confusing... like a large untangled mess where diagnosing one inactive robot, it's not always clear which clause is to blame for the inactivity and which button to press / measure to take to re-stabilize the robot (unless you have something like a complete debug access to RAM + source code). Also it would look a lot like the structure described in the network theory of depression, where many different symptopms and causes are related to each other. Oh, and unless you can directly access the sensitivity threshold (and all other relevant variables), the best thing you can do is maybe access one of the slow-feedback mechanisms and wait for the feedback to kick in after a few weeks.
What does this mean for predictive processing models of depression?
I'm not really sure, but to me, modeling depression looks like a tough thing to do, as there is no obvious "single path that fits all causes" and no clear evolutionary / modelling option that solves all the problems you run when trying to conceive a social robot or whatever. I'm not to sure what to make of this, but as a starting point, I'd be wary of simple, monocausal models for depression...
First, I am curious whether you mean "batch normalization" of artifiacal neural networks, or you actually mean "renormaliztion" which is something different and much more esoteric and unusual.
Batch norm does something like regularize neural network inputs so that all the inputs are rescaled into the same domain of possible values. A loose analogy would be your perception that an object "looks the same" whether it is illuminated by 10 lumens or 10,000 lumens; your eyes and brain have done a normalization for you across a broad range of objective input values.
I'll not from a machine learning standpoint that learning is all about leveraging nonlinearities. If you are routinely going through your network and just kind of smoothing everything out, then algorithmically you are probably making 99% meaningless changes that do very little and 1% terrible changes that effectively destroy some precise, interdependent arrangement of synapse weights.
It might be fun to take those creepy amorphous dog-generating GANs from a few years ago and intentionally do such an operation on them, "averaging down" all the weights, and then observing the result. I am not even all too sure what to predict. It is possible that, instead of seeing a frothing sea of Lovecraftian doggoths, you would get a more vague and less coherent visual image. It's also possible that you would still see some dogs, but maybe fewer. Or maybe you would see a disorganized mass of eyes and muzzles with no structure. Or maybe it wouldn't work at all.
Please link me to Lovecraftian Doggoths! 😂
I think this might be a reference to the DeepDream algorithm that "uses a convolutional neural network to find and enhance patterns in images via algorithming a dream-like hallucinogenic appearance"
See the Wikipedia Article: https://en.wikipedia.org/wiki/DeepDream
I don't know enough about renormalization to know what I mean.
From what you wrote, it sounds like you're most likely talking about regularization, which is where you periodically make all the weights (read: synaptic connections) in your network a little bit weaker. Renormalization is a thing in physics, but I don't know exactly how it would apply to neural networks.
Curiously, I had a comment in the trapped priors thread (https://astralcodexten.substack.com/p/trapped-priors-as-a-basic-problem#comment-1469781) where I analogized how over-regularization kind of looks like a so-called trapped prior, where the system prefers a bad but parsimonious explanation to a more complicated but more explanatory one.
I don't think there's a direct analog between this and the brain, but it's interesting to see something like regularization come up in a physical way.
This doesn’t match my idea of regularization - I think of regularization as a broad class of methods to reduce overfitting. I thought renormalization was a good term for what Scott’s talking about.
But also, batch normalization in NN training is thought to have regularizing effect - so there’s definitely a link between the terms! I’ve even seen a random article or two saying you don’t need dropout if you do batch norm, and dropout is a super popular way to regularize.
Fair enough, I should have specified L1 or L2 regularization.
Did you mean that by the end of the day you have learned three synaptic weights, 0.9, 0.5, and 0.3, and through sleeping the brain does something like normalize them by their sum (1.7) to 0.9/1.7 = 0.53, 0.5/1.7 = 0.29, and 0.3/1.7 = 0.18? That way you incorporate the new observations and over many wake/sleep cycles in theory you dampen the noise because definitionally signal recurs and noise doesn't. I also would have guessed that the right verb is "renormalize".
And then the argument is that some kinds of depression learn fine but normalize lower than a reference bayesian agent would? And therefore symptomatically they diverge most from that reference bayesian in the morning but Central Limit Theorem brings the total synaptic weighting closer to the reference bayesian the longer the person is awake?
And it seems like there are other places you could lower the total weighting, which could look different over the course of the day. If a person learns with less confidence than our reference bayesian, then perhaps they could start chipper but start to drag over the course of the day.
One should be a bit careful to jump too quickly from GANs to brains. It's true that regularization is an important technique in machine learning, and it is fair to ask whether it might have the same effect in the brain.
But there is an important difference. Artificial neural networks are mostly feedforward networks. The brain is a recurrent network. If you forget normalization in a feedforward network, then you get overfitting and a larger testing error. If you forget normalization in a recurrent network, you get an epileptic seizure and die. Reducing overfitting may be a secondary effect of normalization, but the primary effect is "not dying".
Of course, there are some recurrent networks in machine learning as well. But usually the activation function is then designed in such a way that no seizure (uncontrolled explosion of activity) is possible. Strangely, the activation function of real neurons is not like this. It is normal for one neuron to fire at 100 Hz, but if a large fraction of neurons fire at 100 Hz simultaneously, you are close to dying. If excitatory neurons band together too strongly, the best you can hope for is a complete temporary shutdown of the brain area (there are some mechanisms which can do that).
I don't know why evolution didn't choose a different activation function, but the brain relies pretty heavily on homeostasis for not dying.
This, plus experiences of friends and loved ones, definitely makes me wonder if some depression-adjacent sleep deprivation is trying to hold on to/chase the feeling of increasing synapses? Especially because getting enough sleep is something I had always tried to help depressed people with. But Luke comments below, maybe the negative side of sleep deprivation is just the extra-long sleep phases that follow?
Anyway, we can help with the cabin in Alaska, if you're looking to put your six months in.
I think probably not. Most of the time depressed people go to sleep late, it's just because they don't have a lot of willpower or energy and it takes some willpower to make yourself go to sleep on time.
During a few depressive periods I definitely became aware that I was self-medicating with sleep deprivation. I’ve always been a night owl, but when I was depressed I would sort of ride the insomnia and the depression wasn’t as bad the next day. This could be because I had been productive in some way the night before. Or maybe I was just too tired to dwell on depressive impulses. Either way, it was not sustainable and the eventual inability to think straight would make me get some sleep, usually requiring a mild sleep aid.
Definitely a very interesting theory, but I'd be very cautious about buying into it. Just because staying awake makes you feel better and sleep makes you feel worse, doesn't mean sleep is doing something bad for your brain.
There's an analogy with muscles and sleep: If you work out really hard and then get a good night's rest, the next day you'll be sore. If you work out really hard and only sleep a few hours, you often don't get sore the next day. (That sounds counterintuitive, but it's a widely experienced phenomena.) But skipping sleep to avoid soreness is absolutely the wrong thing to do! The soreness is part of the recovery process. If you keep skipping sleep, your muscles will get weaker and weaker as they never recover from your workouts. So, by analogy, I'd worry that skipping sleep to treat depression might be trading off important brain recovery mechanisms.
"doesn't mean sleep is doing something bad for your brain." On the other hand, nobody said that
"It looks like sleep is somehow renewing these people's depressions. As if depression is caused by some injury during sleep"
Apparently Mr Trump holds some rather alternative theories (about having a finite amount of life energy) that have contributed to his minimal-sleep lifestyle.
This leaves me wondering if maybe he has built an unusually stubborn network of synapses that parallels his apparently astronomical levels of dopamine -- making him famous for his astronomically high levels of confidence-in-predictions. Resulting in a character who can absolutely never admit mistakes.
Lord, I wish there was more pruning going on up there!
Somewhat relatedly, the book "Why We Sleep" by Matthew Walker discusses studies that found that REM sleep reduces trauma: in sleep, the person re-lives traumatic memories, but in a detached way due to the emotional system not being fully connected. This causes the memories to be less emotionally-laden and traumatic going forward (or, one might say... "downgraded and renormalized"...)
Those studies might be relevant to the REM / non-REM question.
EMDR mimics the eye movement, and gives a 'disconnected from the Trauma' somewhat trance-like feel.
I'd be vary about relying on much from that book, it's been heavily criticized:
https://guzey.com/books/why-we-sleep/
https://statmodeling.stat.columbia.edu/?s=why+we+sleep
Thanks, that's good to know! Still, the book is unlikely to have made up studies on REM sleep and trauma healing -- so those could be looked up and evaluated.
Having read both the book and the critique, I'm not sure that much of the criticism actually lands. Translating academic research into a popular trade book is always going to involve some level of abstraction and simplification, and the criticism I've read seems mainly to nitpick the particular abstractions and simplifications Walker used when presenting the research.
Like (from memory here) there's a plot he shows with some particular trend. And he chose to omit one single datapoint which lay outside the fit. To me, this seems OK... It's exactly the kind of thing that would throw off a lay reader, while a trained scientist (who understands things like statistical variance, etc) will know it can probably be ignored.
I'm in science communication myself, and the type of things Walker chose to elide or gloss over seem pretty par for the course, if you're presenting research to the public.
This is fascinating. Thank you for these in-depth explorations of science!
Extremely speculative, wondering if others have the same experience:
I often find reading a few highly analytic longposts is the best and only consistent way I can start the day. It seems to actually get my brain "working" while I can't motivate myself to do anything else.
As per the post, the later the day gets, the more energy I have.
But I wonder if reading non-fic essays is more "stimulating" to neurons than say, making breakfast, or going on a walk?
Funny you mention this, but I've had a similar experience. On days I feel very non motivated to work (happens somewhat often), reading cool shit kick-starts my drive to do things.
Do anti-seizure medications sometimes called “mood stabilizers,” like lamotrigine and gabapentin for example, ever treat depression? If they are ever effective in treating depression, not just anxiety or bipolar mania, how does that relate with the parts of this post about seizures, if at all?
Is bipolar depression so different from unipolar depression that anti-seizure medication can treat it effectively whereas unipolar depression can be described as effectively treated in the opposite way?
If anxiety can be a symptom of depression, again, why are another kind of anti-seizure medications, benzodiazepines, so effective at treating it? Admittedly they are described as potentially causing or worsening depression. And often prescribed together with other medications.
This post made me wonder, why is it that anti-seizure medications are ever safe enough and even helpful to give to depressed patients?
But I think I can answer that for myself: there are thresholds, risks, sometimes opposing symptoms and conditions co-occur and need treating simultaneously.
What I am more curious about is: does any kind of depression ever look like a seizure disorder instead of something like the opposite?
Lamotrigine is effective in bipolar depression; its effectiveness in unipolar depression is kind of unclear, possibly leaning towards no (though I will very occasionally use it as a last option). I wouldn't make too much of the "seizures cure depression and antiseizure meds sometimes cure depression" paradox, everything has lots of different mechanisms of action and probably they're effective for different reasons.
Also topiramate makes a lot of people depressed which seems possibly relevant
Gabapentin is also used to treat neuropathic pain. It might be an effective antidepressant in the same way that opioids are...
It also seems relevant that sleep-disordered breathing is common and often goes undiagnosed, yet is often comorbid with depression. Is it possible that some of of the correlation is because people with sleep issues often feel worse when waking, and people with sleep issues are more likely to have depression?
It's kind of hard to find great numbers on this, since at-home sleep studies frequently miss sleep apnea and usually miss other issues, but according to random popular articles (Healthline and similar), about 40% of people with obstructive sleep apnea (OSA) have depressive symptoms, and possibly 15 - 20% of depressed people have undiagnosed OSA. Again, I'd expect these to be underestimates, since the one reference I clicked through (10.1016/j.jpsychires.2019.06.015) relied on really basic at home tests for half of their data.
To me it is crazy that testing for sleep issues is not standard practice for depression given that OSA and other sleep disordered breathing issues are common organic causes for depressive symptoms, and treatment (PAP, surgery, etc.) can improve these symptoms, but that's a bit of an aside.
Testing for sleep issues is hard. The usual process is to screen for them (by asking eg "do you snore?", "do you feel tired after waking?", etc) then test the people who reach some level of suspicion.
It's getting easier. With a Fitbit app, for example, one can view one's sleep on a minute-to-minute graph that pretty accurately tracks time spent in deep sleep, REM, light sleep, etc. So that's a good start (I typically get a horribly deficient 3-5 minutes total of deep sleep).
Then, if you add a modern CPAP (or autopap) into the mix, you can actually layer onto the same timeline a map of all your apneas -- and even discern whether they are likely to be structural or central in nature.
It's also true that Depressive symptoms sometimes look a lot like a sleep disorder, without one being involved. My wife is a sleep tech, and has gotten me two separate sleep studies plus an EEG because of my depression symptoms, because they were legitimately a decent match for sleep apnea of some sort. Everything came back negative though; it's sleep issues caused by depression, not breathing.
Babies sleep a lot, culling and pruning the masses of their synapses. Did depressed patients grow up too fast, or took it too far?
Since a lot of depressed people are (physiologically) full-grown adults, my guess would be the latter.
Depression in small children exists, but is hard to detect. Symptoms are too unspecific and the abilities to introspect and verbalize what they experience is in the range from missing to yet insufficient.
Sleep deprivation leads to ROS accumulation in/on the luminal-side of the gut but not other organs (including the brain). sci-hub.se/10.1016/j.cell.2020.04.049 This is evident in experiments with mice and flies after even two days of deprivation. Graphically evident with photos as in Fig 4. With longer deprivation, death results. If the luminal-side ROS is discharged through the by-mouth use of various compounds such as NAC, melatonin and lipoic acid, the animal is rescued from death even with sleep deprivation. Recovery sleep dissipates ROS accumulation in the gut. The effect of sleep deprivation is not all (or ?even?) in the brain.
The gut microbiota influences the brain and behavior through, among other things, the vagal nerve. sci-hub.se/10.1016/j.neuron.2019.02.008 Preventing vagal communication between the GI tract and the brain increased occurrences of psychiatric-related disorders. Vagal nerve stimulation can substantially improve MDD. Gut bacteria also influence the brain and psychiatric disorders through the vagal nerve. The vagal nerve has about 80% afferent (gut to brain) and 20% efferent (brain to gut) fibers.
Hypothesis: If the depression is caused by a specific gut ecosystem signaling, suppressing the signal between the gut and the brain through ROS accumulation in the gut could suppress the depressive symptoms. As demonstrated in the first reference, sleep dissipates the deprivation induced ROS. A possible test of this hypothesis would be to have patients whose depression are relieved by sleep deprivation take one of the tested supplements (e.g. lipoic acid, NAC, et.al.) and test for continued depression relief with sleep deprivation.
The stool transplant as a psychiatric treatment will be not so strange an idea as it is now. (Source: vivid imagination)
+1*
Thanks for pointing out the connection between sleep deprivation and gut ecosystem (which I hadn't thought about before)! In particular, since "if the ROS[the thing that accumulates during sleep deprivation] is discharged..., the animal is rescued from death even with sleep deprivation", this makes me think that depression might be a kind of way of being alive, because becoming less depressed by staying up eventually kills you? (But I also notice that non-depressed people typically stay non-depressed after going to sleep for a night and waking up in the morning, contra an intuitive model that of sleep & depression in which each sleep is a Big Depressing Injury, so there must be something about depression that makes sleep depressing for depressed people but non-depressing for non-depressed people**.) Either way this sounds like a beautiful and highly plausible theory. Also since NAC/melatonin/lipoic acid clear ROS, this suggests that either those or their pharmacological opposites should be depression treatments.
*Is this not allowed anymore as a form of pseudo-upvoting? If so, sorry and please ignore.
**And now I've reached semantic satiation for words that begin with "depress". Congratulations.
I guess this theory can explain your problem of trapped priors: Whenever a small not-so-negative experience (eg a only slightly aggressive dog) happens it gets deleted by a faulty renormalization afterwards and only a clear positive experience (eg a cute puppy in a cage) can help. Apparently, a correct renormalization would keep experiences of the first kind, but for depressed people it is more likely to throw small updates away (since their brains have less connections overall).
Maybe I'm melancholic depressive, though I don't know what that means. Any time I wake up at maybe 6-8 am, I'm suicidally depressed. The first thing I think on waking is, "Dammit, I'm still alive." But I know the feeling will dissipate in an hour or two, so I try to ignore it.
This happens regardless of my sleep cycle. If I go to bed around 3am and get up between 10 and 11, I have no depression at all.
(I haven't noticed whether it varies with season. I don't have natural optimal sleep-wake times; I have a natural daily cycle of 25 hours, so my sleep cycle moves forward 1 hour each day until I force myself to get up early and suffer a sleep-deprived day to restart the cycle.)
~25 hrs is normal in humans, it is reset daily by light cues. A strong light to wake up with (see the Lorien post on light therapy) could be worth a try if you haven't already.
If 25 hrs is normal, then humans must have evolved 200 million years in the future. Probably the Doctor was involved.
Thanks for the pointer, but I'll probably just keep doing what I'm doing now: always sleeping until after sunrise. I wonder whether Seasonal Affective Disorder isn't seasonal at all, except as an artifact of clocks. Just a bunch of people who get depressed in the morning twilight, but notice it only in winter because nowadays we get up at the same clock hour year-round rather than getting up when the sun rises.
Thanks for bringing this up. I was also under the “natural is 25 hrs” misapprehension, and went to look at https://en.m.wikipedia.org/wiki/Circadian_rhythm#Humans to look it up and paste here, and apparently that study was bad and after they redid it it’s much closer to 24 hrs
TIL. Thanks.
This corresponds precisely to my experience. I never thought about this systemically before, but I have always struggled with getting out of bed in the morning due to an overwhelming feeling of melancholic depression, exacerbated by the fact that I stay up later than would be wise for my own good because I usually feel most at peace with the world and therefore energetic at night. The occasional time I manage to stay up all night, I get to work the next day with verve and a more positive outlook, partly because of a feeling of accomplishment I suppose. Unfortunately there does not see to be any apparent solution to this, seeing as eventually Morpheus does call for his due.
Consider reading the chronotherapeutics manual linked in the first paragraph; it may have some helpful suggestions.
It looks like a few people here are relating this blog to the one on trapped priors. Eric Hoel has some work (e.g. https://arxiv.org/pdf/2007.09560.pdf) on "The Overfitted Brain: Dreams evolved to assist generalization" that seems relevant. His theory (which he presents some data in support of but I don't have enough knowledge of the field to know how convincing it is) is basically that the brain, like a neural net, is prone to overfitting based on the data set that it encounters. And in neural nets they often solve that problem by adding noisy or corrupted inputs, which make the fit worse on the existing data but make the system overall more capable of being generalizable and adaptable. And that dreams are the human equivalent of that. From the abstract: "Dreams are a biological mechanism for increasing generalizability via the creation of corrupted sensory inputs from stochastic activity across the hierarchy of neural structures. Sleep loss, specifically dream loss, leads to an overfitted brain that can still memorize and learn but fails to generalize appropriately." I don't think I can quite put my finger on the model that connects them (maybe because it doesn't exist) but there seems to be a strong common thread between depressed brains suppressing new input, and dreams purported function in preserving openness/flexibility, and sleep being messed up in depression.
I wonder if this explains my experience, which is that I feel like I undergo a steady uptick in anxiety the longer I stay awake, which I think is a fairly general phenomenon, ie "your brain at 2am suddenly bombarding you with restless thoughts."
I wanted to ask. I suffer from anxiety and find that my anxiety is worst in the morning, by far. Does anyone else experience this?
(As in, anxiety rather than depression)
Yes, I think I do as well.
Yes. On the first deep inhalation (the "now I'm awake" moment) anxiety explodes (on bad days). It can take hours sometimes to come down and get functional. After falling asleep again it can happen again, even if the first waking up has been OK. I tried to see if that changes but up to four times (couldn't get back to sleep more often) it comes back.
So what does waking up around 3AM, often associated with depression, do in this model? Is some failsafe processing in the brain detecting "too much" synapse clearing and waking you up to stop it?
>unsurprisingly SSRI antidepressants decrease REM sleep a lot (not just in depressed people, in >everybody).
I found this pretty surprising and would be interested to hear more. How significant is the decrease? how dose-dependent is it? etc.
Background:
I have had insomnia for basically my entire life which could charitably be described as "severe" and uncharitably described as "Holy F****ing H*** how are you still alive?!?!" I take a few different meds and supplements for it, in addition to sleep hygiene and other lifestyle interventions, but the main medication is Trazadone, an SSRI, and when my sleep specialist first prescribed it after my sleep study (which was apparently pretty horrific looking, even to someone who dealt with severe insomnia cases every day) she said it had been shown to help produce deeper more restful sleep, rather than merely helping with sleep onset. This was important because a big part of my problem was not getting any rest even when I theoretically slept; my sleep study showed (IIRC, it was almost 20 years ago) something like 5 hours 'asleep' out of 9 hours in bed, only about 20 minutes of that REM, and zero slow-wave sleep, I was just staying in the first stage of sleep all night, when I was asleep at all. No one sleeps well during a sleep study, you are in a strange room with electrodes taped all over you, but this was bad even accounting for that. I guess I am rambling quite a bit now, but my point is I was very surprised to hear SSRIs would /reduce/ REM considering what my sleep specialist said back in the day, and how well it seems to help me given how I sleep when I am not on it.
Trazodone isn't an SSRI.
Oops. you're right of course, it is one of the older (non-selective I guess?) serotonin re-uptake inhibitors. For some reason I was remembering it being an early SSRI. But if the decrease in REM is caused by increased serotonin wouldn't you expect the same effect? I'm afraid I've been on it long enough I've forgotten most of the reading up on it I did initially and have regressed to "as long as it keeps working I'll keep taking it"
I might be wrong about this, but I don't think trazodone reuptakes serotonin at all. I think it is mostly just an agonist or antagonist at various serotonin receptors.
I'm seeing it classed as a serotonin antagonist and reuptake inhibitor (SARI), but not much on the relative strength of actions, so it is possible the reuptake inhibition is simply insignificant.
Low confidence in predictions may be an accurate read of reality in the face of a stressful or unpredictable environment. In this way it could be an adaptation to stress which could also include social withdrawal, apathy, and pessimism, or seen differently a switch into a mode of conservation and survival as opposed to a mode of exploration and expansiveness.
The amount of sleep that people have goes down with age, until retirement when it starts going up again.[0] Definitely some of that is due to the amount that they're able to sleep (a person with a job and two kids is going to have very little sleep), but is any of it related to different needs?
[0]: https://www.bls.gov/tus/charts/sleep.htm
For an economic perspective on this, see Daniel Hamermesh's *Spending Time*.
Here is a review from the Subreddit:
https://www.reddit.com/r/slatestarcodex/comments/f0wsr8/book_review_spending_time_by_daniel_s_hamermesh/
> Consider this observation: high-income people spend less time sleeping and less time watching TV than low-income people. Why is this? “That’s obvious,” you might think: “lazy people spend their time sleeping and watching TV. Since they don’t work as much, they don’t have a high income.“
> Is there another explanation? Hamermesh suggests this one: everyone has to spend their time - we all get the same 24 hours each day. High-income people have the option of spending it on expensive things, like eating at nice restaurants and working out at fancy gyms. But low-income people can only spend it on cheap activities, like sleeping and watching TV. Put another way: if you have more resources, you face higher opportunity costs.
> The time use data allows for these theories to be distinguished. Hamermesh looks at people who have lots of income because they work (i.e., they have a high-paying job) and people who have lots of income for some other reason (e.g., their spouse works a lot, they won the lottery). And he finds that the second theory is better: the high-income people who don’t work aren’t spending much time sleeping or watching TV. So (on average) it’s high income -> less sleep/TV, not less sleep/TV -> high income.
This pattern - high sleep in youth and retirement, low sleep in prime earning years - could match Hamermesh's view.
That chart explains nothing about what's going on because it doesn't distinguish between weekdays and weekends.
Of course, some people get to sleep as much as they need, but I would estimate that a lot more people are chronically underslept on weekdays because of work and kids. On the weekends the underslept people try to catch up on sleep but do not always succeed.
I wonder if they have a more useful chart somewhere. I couldn't find it.
Has anyone ever tried sleep reduction therapy to treat depression? If avoiding sleep entirely seems to work until you go back to sleep, what happens if you only sleep 4 hours a night? How long can a human go with artificially reduced sleep?
What about going the other way? If we make people take sleeping pills to sleep more, do they develop depression?
For both questions, does changing sleeping patterns eventually cause the brain to renormalize to that level of sleeping?
Having been in the military in an active combat unit in an active combat role, even just for training purposes, the answer to "how long" is a very long time with sufficient motivation. I can't say the effects are in any way positive, but granted, there are a lot of confounders. Even when it's just Ranger School and you're not actually in danger, there are plenty of sources of stress aside from just the lack of sleep.
There are the ultra rare cases of people who just genetically need less sleep, including the famous Al Herpin: https://en.wikipedia.org/wiki/Al_Herpin. Wikipedia seems to not believe his claim to have never slept, but if he were telling the truth, would he be the happiest man in history?
Sleeping less than my body wants to helps with mine. 4h would be excessive, I think I average 5-6h recently, while the "natural" amount would be around ~9h.
Even if I crash on a weekend day as the body wants to catch up with the sleep debt, that's one day with depression vs an entire week with depression.
Perhaps it is an under stimulation problem? You sort of allude to this in the post, but do not really explore that possibility further.
I basically fit this description to a tee. I have some intermittent problems with depression. And sleep deprivation removes it completely. Periods of slowly slipping into depression followed by sleep deprivation to snap me out of it used to be a recurring pattern for me. But I notice that when I force myself to get seriously mentally stimulated (reading some difficult texts, doing math problems), the depression goes away. Or is less severe. Especially when done earlier in the day. Caveat though, it is not major depression for me, although suicide has occasionally occurred to me in the past.
And actually working too hard, spending too much time and mental energy during the day gets me into mildly manic moods where I have a hard time falling asleep. Usually followed by tiredness where I spend less time doing difficult tasks, where I slowly slip into depression again.
This is so effective for me, that when I get depressed I know I have been lazy and need to improve my work ethic. It is kind of annoying that this only works for tasks that I do not easily pick up, especially when depressed. So once I slip into a depression it can be hard to get out of it without sleep deprivation.
So each person might have different needs for mental stimulation and sleep. And perhaps sleep deprivation to snap out of it, followed by a constant diet of doing mentally taxing and rewarding tasks is the cure for people who suffer from this. It helps if you set an overarching and at least somewhat ambitious goal that you can get excited about.
It seems like a reasonable inference from this is that someone should try treating depressed people with small doses of melatonin, like it's a sleep phase disorder. Does anyone know if this has been tried? It's in no pharma company's interest to find out if melatonin can treat depression so probably not, but maybe that fancy patented melatonin that there was an SSC post about once.
Also, what about depression with atypical features? It's definitely not rare. So it seems like there are also a lot of depressed people who feel *better* after sleeping, and I wonder how that works with this theory.
Phase shift with light therapy (and sometimes melatonin) is moderately effective for depression. A recent study showed the combination of melatonin+buspirone seems effective too.
I don't know how atypical depression fits in here.
If this type of depression makes you wake up too early, do you have people take melatonin first thing in the morning (like you suggested for advanced sleep phase disorder)?
What doages and timing for melatonin works?
Not for depression specifically but here's Scott's long post about melatonin and timing for other indications https://slatestarcodex.com/2018/07/10/melatonin-much-more-than-you-wanted-to-know/
Three observations come to mind:
1. "You can't renormalize while the network is running." Says who? This seems like it might be a case of unconsciously extrapolating from *human* designed machines to natural machines. Human beings, for reasons of design and construction efficiency, do indeed design machines that generally need to be shut down in order to be worked on -- we have to close the road to repave it, we have to shut down the computer to replace its graphics card.
But I would say as a rule biochemistry tends not to operate this way -- there tend to be many multiple overlapping and redundant paths for getting the same, or similar things, done, and it appears pretty common for the body to work on a system at the same time as it operates. Bones don't grow longer (in children) only when they are not bearing weight, we don't heal wounds and injuries better or faster when we don't use the injured area, and we don't always choose to run only catabolic or anabolic pathways for the same substrate.
So I would say this idea should actually need to be measured to be sure it is true, especially since it's a key assumption underlying the idea that retuning the synapses needs to be done during sleep.
2. Why is it said depression gets "better" when one is sleep deprived? Maybe sleep deprivation merely adds a temporary cover layer of mania that hides the depression, as if you gave the depressed person a massive dose of amphetamines. That is, the change may not be in the direction of cure, but in the direction of adding *another* layer of brain dysfunction that conceals the original layer somewhat, so, away from a cure. If so, the last thing we'd want to do is try to extend it.
3. If a *general* synapse realignment underlay depression, why wouldn't there be much broader associated brain dysfunction? That is, shouldn't depressed people also have noticeably lower IQ, poorer executive function, worse memory -- even compromised sensory abilities and downgraded motor skills? That is, shouldn't an across-the-board physical degradation of the brain have across-the-board degrading effect on brain function, from soup to nuts? Why would we see the primary symptom in mood (which seems a fairly high-level complex epiphenomenon of neural function) rather than in, say, memory impairment or crappy motor skills, something more basal?
Depressed people do have lowerIQ, poorer executive function, worse memory, compromised sensory abilities, and downgraded motor skills (look up eg "psychomotor retardation", "pseudo-dementia", and the research on depression and olfaction). It's usually pretty subtle, but can be obvious in sufficiently severe depression.
Shouldn't it be quite hard to tease out the effects of simply being less motivated to take tests and succeed at motor tasks from actually being physiologically worse? I can see testing purely automatic reflexes for sensorimotor response (but not control) but can't even think of how you would try to compare intellect across test takers trying to do well and test takers who aren't trying to do well. Memory maybe, but how much is worse memory and how much is paying less attention because you don't care enough about what's happening to want to remember it?
"Maybe sleep deprivation merely adds a temporary cover layer of mania that hides the depression, as if you gave the depressed person a massive dose of amphetamines."
I'm inclined to see that as the way it works, particularly since the depression comes back once the sleep deprivation stops. The sleep deprivation quoted isn't merely "skip a few hours sleep", it's "long enough to start bouncing off the walls" periods.
On point 2: I'm thinking that in some cases, "a dash of mania" might just be the same as "a reduction in depression", and in other cases not. Depression is heterogeneous, and some depressions look and act more like mirror-mania than others. There's a type of bipolar episode called a "mixed" state, where symptoms of mania and depression co-occur--so there are at least some cases of depression that aren't cancelled-out or masked by manic symptoms. In these cases you see a lot of agitation, anxiety, irritability. I have to wonder what kinds of effects sleep-deprivation has on the 30% of non-responders. (Interesting to note that treatment-resistant depression may be stealth bipolar in a lot of cases, and you'd expect a population like that to have more sleep-deprivation responders than in depressives broadly, where I want to say the rate is more like 50%.)
waitaminute... one model of autism is that autistic people have an overabundance of synaptic connections (https://www.cell.com/neuron/fulltext/S0896-6273(14)00651-5). And autistic people get less REM sleep (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3111973/). Is Autism a SLEEP DISORDER!?
I think there's something in that. I've heard they tend to produce less melatonin naturally.
My daughter has an ASD diagnosis. She used to be very troubled at school, and also slept badly. Since she started taking melatonin, her sleep has improved a lot, and she's had a massive reduction in meltdowns and challenging behaviour, to the extent that I'm not sure she'd get a diagnosis now. (She still has the "good" or "neutral" autistic traits, like systematic thinking, good memory, and preferring her own company.)
Speaking for myself, I don't know whether I'm on the spectrum or not (several traits, genetic link, but never been formally tested), but I know that if I don't get enough sleep I'm much more prone to sensory overload, so I'm guessing her experience is similar.
(although, usual disclaimer about correlation not equalling causation; she might just have grown out of the meltdowns with maturity)
Yeah, this aligns with some of my own anecdotal experience; I slept horribly as a kid and my autism symptoms improved substantially in my mid-20's around the time that my sleep improved too.
There are some ML failure modes this brings to mind as potentially useful ideas here.
Sparse gradients and local minima effectively make improving a model impossible. In practice, this usually means you throw the model away in start over. But you can't do that with a human, so you need some way to escape from the local minima. Stronger gradients are one way to potentially do that - a steeper gradient will tend to push what would have been smaller updates into bigger ones. The downside of taking bigger steps like this is that you often end up in a worse place than you started, because bigger steps means that the local gradient is less predictive of how good your new state is going to be.
But if your problem is escaping a local minima, that can actually be a very good thing! Even if your immediate situation is worse, you *might* now have a path to improvement, since even if your new state sucks, *it might be next to a place that doesn't suck*, and that might be next to a place that sucks *even less*, and there might be a path through state space all the way to an actually decent model.
More generally, a lot of depression treatments looks awfully similar to the kind of desperate tricks you might use to get a model out of a local minima.
"More generally, a lot of depression treatments looks awfully similar to the kind of desperate tricks you might use to get a model out of a local minima."
Tell me more about this?
SSRIs and bupropion both effectively increase the signal provided by various neurotransmitters. Assuming that those transmitters are involved in the feedback loops that do some kind of gradient descent in biological neural networks, more signal = more gradient = bigger steps = better shot of escaping local minima. Sleep deprivation, in addition to the mechanisms you describe in this post, also just breaks a lot of feedback mechanisms things - usually bad, but great if you just want vastly increase the variance in feedback to escape a local minima. Electroshock therapy seems to basically be throwing a ton of (nonmeaningful) signal that knocks things out of whatever steady state they're currently in and kicks you some ways away in state space to some point that may itself suck, but has a better shot of having a gradient you can descend. Exercise generates a lot of signal. Changing your environment generates a lot of signal. Social interactions are complicated and signalful. Psychadelics radically alter whatever state your brain's in - and wherever it ends up, it's less likely to be a local minima than wherever you started.
The general testable prediction here is "increasing feedback to generate steeper feedback gradients or otherwise significantly altering the state the brain is in, pretty much no matter how you do it, has a chance of escaping the local minima of depression. More intensive feedback/more aggressive state shifts are more likely to be effective than less aggressive mechanisms. You're not aiming for improvement - just change to a point where improvement may be possible."
Sorry, you probably wanted me to elaborate on the ML side of this. A lot of ML failure modes center around sparse gradients - basically, gradient descent failing because all of the reachable points in state space are worse than where you are. You're in a valley in N dimensional space, you can't go downhill.
In industry, the usual practice is to throw away the model and start over. But throwing a ton of signal/increasing the gradient to explore the surrounding space/randomly perturbing weights are the sorts of things you would try if that wasn't an option. You can sort of see this in guides on how to avoid e.g. mode collapse in GANs: https://github.com/soumith/ganhacks . A lot of the advice is based on the idea that you really want to keep generating some kind of signal, and noise and randomness keep popping up as mechanism to induce that.
If it helps (Scott, not you, as you obviously understand this quite well), I think you can understand how dimensionality contributes to this by obvious analogy with normal space. If you constrain yourself to moving in a perfectly straight line, your options are pretty limited in where you can go. Open up to a second dimension and suddenly you can rotate 360 degrees and have infinitely many options just in the choice of direction. Open up to three dimensions, and well, think of how volume scales compared to area. Suddenly, the search space is a lot larger. This extends quite intuitively to extremely large dimensional spaces. The search space is absolutely unimaginably enormous and the fact that we find reasonably good solutions anyway is nothing short of a mathematical miracle.
This is exactly why regularization works so well. When you have so many dimensions of data, the overwhelming majority of them are insignificant, and the Bayesian optimal search assigns little or even zero weight to most of them.
I have no idea if this sort of observation analogizes in any way to human brain anatomy, though, as it suggests a good treatment strategy when the brain goes wrong is to just willy nilly destroy synapses at random, figuring most of them aren't doing any good anyway.
I mean, the sort-of-surprising-thing in this context is that you can have 500 million dimensions and still have basically no slope (either a local minima or a flattish valley). This make sense when you switch back to thinking about it as an optimization problem, because intuitively reasoning about high-dimensional spaces is hard (e.g. high-dimensional oranges are almost all rind)
> I mean, the sort-of-surprising-thing in this context is that you can have 500 million dimensions and still have basically no slope (either a local minima or a flattish valley).
It makes more sense if you stop thinking of it geometrically and start thinking of statespaces.
If I have a processor with 64MiB of memory... sure, it's got a state space of >500m dimensions, but you can still easily come up with local optima for a lot of objective functions.
For one example... if we're using our 64MiB of memory as the input to an optimizer that's trying to successfully execute an instruction at the highest address possible, a nop slide across all of memory is a fairly obvious local minima. You can do better, by remapping some physical address to the end of the virtual address space and executing it, but that requires setting up page tables / etc / etc, none of which is going to help you in this local state.
Of course this replaces one question with another (why do things start looking statespace-like?). And glosses over discrete versus quasi-continuous variables.
Sorry if this didn't end up being very helpful - I was rushing and glossing over a lot. I might try to expand this into a post later.
Has anyone tried just putting a depressed person into a really exciting or unusual or terrifying/thrilling (but safe) situation? Like taking them to an amusement park and making them ride on all the scariest coasters and rides?
Families try this a lot, this is the standard family let's help out the depressed guy strategy, especially before diagnosis but even after diagnosis, families usually will keep trying this periodically.
In my experience, this has diminishing returns, that is, if someone has not had something enjoyable for a while, the impact of having an especially enjoyable and exciting experience can be significant. However, if the depression is persistent, trying again and again to throw enjoyable experiences at someone eventually becomes irritating and exhausting and reinforces a feeling that really enjoyable experiences are out of reach. In really deep depression, sometimes the depressed person will barely be aware of the enjoyable experience even as they go through the motions, since their attention will be on the unrelenting drama in their skulls.
In short, it's a good idea, worth a try, but not a sure-fire cure.
I hate heights, I've been on a Ferris wheel once in my life and that was more than enough, and if you forced me onto a roller coaster not alone would I come out still depressed, I'd come out depressed *and* unmercifully pissed-off so I hope you have a good dentist because you'll need them because I'm gonna knock your teeth down your throat
My wife is depressed (on treatment and doing fine) and she *loves* roller coasters. Days we can go to amusement parks rank among her favorites. She's speculated that if she could just ride roller coasters on the regular, she might not need Zoloft.
If any strong, random perturbation has some reasonable chance of knocking someone out of depression, you have you wonder why more people don't just stumble on depression cures going about life, getting almost hit by cars or tasting ghost peppers for the first time. A lot of depression does spontaneously remit, so this could be happening, but my thought is that within a depressed states, just submitting yourself to the right kind of stimuli is not going to be enough to get you out of the local minima--the problem is that you aren't interpreting signals well, not that you aren't getting getting the right kind, and everything is just more depression-fodder. A signal strong enough to shock you out a local minima depression would have to be something that the depressed model of the world just was not able to account for or incorporate--otherwise you just roll back down the hill, no?
Rather than stronger signals, I think the commonality in depression treatments under the local minima theory would be things that alter the updating process itself. I think that would explain the lag between treatment and improvement, actually--it takes awhile to slowly grow out of a maladapative network, and changing the weight updating procedures just kicks that process off. If a person were just settling into a new pattern after a strong shock, you'd expect rapid improvements instead.
The reason I asked about roller coasters is because the "any strong shock" theory does run true to my experience. Granted it is just anecdote, but the one time I suffered from depression, a completely random and surprising experience *did* seem to immediately knock me out of my depressive state. I was sitting on a bench, literally thinking to myself how I really wished my life would end, but didn't have the energy to be bothered to do anything about it, when suddenly a very, very old friend who had moved far away, that I had not seen in years and never expected to see ever again, happened to walk by. It was completely shocking and surprising, and after we had a brief conversation exclaiming about how weird it was, I found that I felt 100% back to "normal" and no longer depressed. Now, this depression had only lasted for about a month, and seemed to be situationally induced after I had been sick and forced to stay in bed for several days. So it was not an ongoing chronic depression, and perhaps wasn't that "deep", though it had been unrelenting for a month. But in that case, after the shock occurred, it was like I was literally jolted into a different frame of mind and back to normal. I felt instantly better and the depressed feelings didn't return. So the strong stimulus theory rings true to me, at least for more minor or short-term depression. Of course, it's not easy to manufacture an artificial but effective "shock"....I thought a stimulus that ramps up adrenaline and emotion, like a rollercoaster, might work.
I think we actually agree a lot more than you think. The word "strong" is doing a lot of work here, and I think unpacking it dissolves some of the apparent disagreement.
A common way to stumble into local minima is with vanishing gradients - coefficients so near to zero as makes no difference. You might end up in this situation if the reward feedback mechanism for a given system consistently couldn't make use of some piece of information and so updated to treat it as noise. If that happens too often, eventually the system is going to treat everything as noise. And once that happens, you're stuck. To pick a more specific failure more: If the first layer of your network just multiplies all of the input by 0, the rest of your network can never learn anything.
To a network beset by aggressively vanishing gradients between the input and most of the network, what most systems would react to as a strong signal would elicit the same response as a weak signal.
So, you can try perturbing the weights, forcing the network to start processing information again, and hope that this time you get a little luckier and it finds some way, even if it's small at first, to get some reward out of some of the input. Or - maybe there are input channels that are usually zero, so the weights escaped the great purge of noise reduction that killed everything else. Parts of the network that never played a significant role in generating the output that the loss function penalized so aggressively, and so never got back propagated into oblivion.
If you could generate some input on those unused channels, and the non-zero parts of the network find some way to marginally update the weights that improves performance - well, then you just might be in business. You can start descending some gradient. And then, the nodes that take input from the used and unused channels - well, maybe it turns out that some of them actually can make use of some of that previously-zeroed-out input, so it updates the input weight from there from 0.0001 to 0.0003, and then maybe 0.001, and then it turns out that you can combine that with some other input to improve performance a little more...
In the hill-descending analogy, this is equivalent to thinking that you're in a valley, and then suddenly your third eye opens and you realize that even though north, south, east, and west are no good, east-by-wourth-blorgward *does* go downhill
Electroconvulsive therapy seems a lot like this. It's effectively scrambling all of the weights and resetting network state. You can't really reset a human without inflicting actual brain damage, but we certainly try.
Yeah. I wish I knew more about the details of what ECT actually does to the brain, but from my layman's perch it looks a hell of a lot like aggressive random weight perturbation.
Having been through ECT, what the long-term effects felt like to me was having the connections between memories disrupted. (As well as the connections between aspects of what one might consider a single "memory".) It's been about 14 months since I stopped, and I'm still getting odd deja vu. My impression is that most or all of the data is still there, but how would I know? Sometimes I look at things I did or wrote in the past, and I can't connect to them at all. And then sometimes some other seemingly-tangential event will bring back some of those memories, memories that I didn't get from simply seeing the thing itself. And there's always the nagging worry in the back of my mind that I'm just fabricating the "new" memories on the fly.
The short-term effects of ECT were like all the pleasant parts of being drunk, without any of the unpleasant parts. It was quite nice. I'd come out extremely happy and smiley and dopey, and open and trusting and joyful to see whoever it was who came to pick me up. Then over then next 24-36 hours, I'd slowly start to re-connect with all the reasons I was depressed. My go-to literary analogy is from Bujold's "Komarr", when Ekaterin goes under fast-penta, and then comes out of it.
R. Pirsig described the effect of excessive ECT, an erasure of the personality and the subsequent encounters with the ghosts of memories, in "Zen and the Art of Motorcycle Maintenance".
In case you have read it, does his literary description match your experience in quality (obviously not in quantity or intensity)?
I read it a while back, but it's now on my to-re-read list for obvious reasons. :-) From what I recall, the fictionalized depiction in the book doesn't match up at all with what happened to me. I think my personality is largely the same, but the ECT got me out of a severe paralyzing depression. You know that thing in depression where thought kinda spirals around a black hole, and anything different you can think of immediately relates back to the depression somehow, and just drags another chunk of your life into the pit? It feels like that effect got disrupted, along with a lot of the connections in my memories. I think I'm better at holding on to the good bits, now. If that makes sense?
However, there are a lot of variations in how ECT is done and has been done, and some of the changes have been in direct response to problems people had, or to abuses like the ones depicted in the fictional "One Flew Over The Cuckoo's Nest". The current is applied to different parts of the brain, there's different types of current, and there's also a lot of attention paid to the frequency of treatment. The staff were the absolute nicest I've seen at a hospital (not that I have that much experience, thankfully). Even the modern name "electro-convulsive therapy" was chosen to move away from the word "shock". So while my experience in 2019-2020 was different than what was in the book, I don't think that has much connection to whatever actually happened to Pirsig in the 1960s. Maybe his experience was a lot closer to what he wrote.
Good to hear you got out from orbiting the black hole.
Quick google: Pirsig's ECT was real, but exaggerated, according to this paper: https://pubmed.ncbi.nlm.nih.gov/19201545/
> " newly published biographical information on Pirsig from Mark Richardson (Zen and now: on the trail of Robert Pirsig and the Art of Motorcycle Maintenance. New York: Knopf; 2008) has documented that the role of ECT in ZAMM is a 'literary device', added at a late stage in drafting the book. In reality the ECT had erased some short-term memory, but Pirsig's long-term memory had quickly returned. Richardson obtained this information from Robert Pirsig's (then) wife, from his sister, and also from his friend John Sutherland (who appears as a character in ZAMM). It seems that one of the most famous depictions of ECT, one that had appeared factual, was actually fictional."
So, not fully fictional, just massively exaggerated.
More generally, my guess is that ECT, like hypnosis, 'truth serum' (and other candidates?), are (or were) understood badly enough by the general public of their literary and artistic heyday to be an easy tool for authors to create a thrill, as a means for the (usually evil) adversary to exert extraordinary power.
What do you mean by this? After ECT most people eventually recover and have the same memories as before - why would this happen if you changed all the weights?
I realize that you weren't replying to me, but... what do you mean by "most"? I haven't done any formal review of the literature or anything, but my impression was that permanent memory loss was a common side effect, affecting up to 1/3rd of ECT patients at a minimum. (Varying depending on lots of things, including the exact type of ECT treatment used.)
So there's a few different things this brings to mind, which I'm going to list off in no particular order:
1. You are changing all of the weights, but it's important to remember that it's a perturbation, not a totally random reassignment. State space is Extremely Very Very Very Very Very Vast, and in the grand scheme of things you haven't moved *that* far. That means that *most* of the information you had before is still there.
2. Human brains and many large ML neural networks often contain a lot of redundancy. You can observe this in action with "knowledge distillation", the umbrella term for methods of closely approximating the results of a large, expensive model with a smaller, cheaper one - even though the distilled model contains a fraction of the raw information as the original, it still seems to "know" about as much. Most of the big NLP models you've heard of, like BERT and GPT-2/3, have distilled versions that perform very very close to the big version using a fraction of the resources.
3. Aggressively perturbing a model is likely to break it in some ways - this is the price we pay for our local minima escape attempt. But the things that *worked* about the local minima are likely to be recovered in short order via gradient descent. They're relatively close in state space, and it's likely that following the gradient from our new perturbed location will converge with our previous local minima along many dimensions (meaning a lot of the weights are likely to return to very near what they were before we shook everything up). The hope is that *some* of the dimensions diverge such we don't end up back in the same valley we were in before, and we can continue descending the gradient without getting stuck on a local minima.
4. I'm guessing that some machinery involved with recovery memories gets damaged by ECT, but that this is the sort of thing that's likely to get recovered by gradient descent (since successfully remembering things is "rewarded").
Scott's is an interesting theory but it does not hold. People with schizophrenia also have lower synaptic density (see https://www.nature.com/articles/s41467-019-14122-0 ) but so far nobody noticed that sleep deprivation helps with schizophrenia.
Secondly, my guess would that the negative correlation with the depression score will turn out to be spurious when we get a publication with a larger sample. The sample in the article Scott quotes is very small (n=40).
Schizophrenics also have very disturbed circadian rhythms, and the negative symptoms of schizophrenia can look a lot like persistent depression and are very difficult to treat. Since you can't treat negative symptoms in schizophrenia very well with other depression interventions, I wouldn't take the fact that an intervention works for some depressed people but doesn't work for schizophrenia as a sign of the goodness of fit of a theory about the etiology of depression.
On a lighter note, psilocybin increases synaptic density after a single administration (in pigs) https://www.mdpi.com/1422-0067/22/2/835 - Looks like we have a new therapy for schizophrenia and depression. And also you do not need to sleep, if you take some psilocybin, if Scott is right.
That means psilocybin will cure depression :) Now - to schizophrenia ;)
Or you'll still be depressed, but you won't notice because you're so engrossed in your conversations with the cactus people 😁
That's really surprising to me, as I would have thought psychedelics released us from all those well-structured connections -- much like the dream state.
>How does this relate to TMS?
The #1 most common side effect of TMS depression protocols is extremely vivid dreams, by a long shot. I have had one of these protocols and can attest, it's friggin crazy. If it's non-REM sleep that's doing harm, maybe TMS increases not only the intensity of dreams but the amount of REM sleep.
Most antidepressants (in multiple different classes) also frequently cause extremely vivid dreams, especially early on in treatment, so I wonder if there's some similar sleep mechanism somehow
Convenient timing - the Royal Institution in England recently hosted an online talk by Gina Poe on this subject:
https://www.eventbrite.co.uk/e/essential-functions-of-sleep-learning-memory-and-changing-your-mind-tickets-138758722073
These usually appear on their youtube channel (https://www.youtube.com/user/TheRoyalInstitution) in a month or so, so it might be worth keeping an eye out for that.
Gina actually described three stages, REM, deep sleep, and a "transition period".
Deep sleep is the physical repair time, when the brain is least active.
REM and the transition period both are connected to the rewiring of your synapses, but one strengthens new connections, the other clears out unneeded connections. Or one organises, the other files/dumps depending on the results of the organising. Figuring out exactly what happens is obviously an area of ongoing research.
A title reminded me of the exurb1a's video: https://www.youtube.com/watch?v=-mu780uB7mI
Two interesting notes that might help along or that you may find extremely boring:
* Donepezil is a great way to cause more REM, other drugs that act in complex ways on the cholinergic system do too. Looking at trip report of these might be interesting. The big problem here is that most people taking these REM-increasing drugs do so during the day and have serious conditions (e.g. alzhimers)
* REM and memory consolidation are potentially inversely related (not sure if this is what you mean by "sleep renormalizes the hippocampus"), or rather, REM and short term memory consolidation may be, at least in that N2 and N3 seem to play a big role there and increase N2 and N3 seem to => short term memory consolidation. And I'd usually associate a longer REM with a longer N4.
* Pulse oximeters can kinda-maybe measure sleep stages well enough to be not noise and so can some comfortable 2-4 neuron EEGs. These will never be used in a study *but* trying to get feedback on your sleeping patterns and what helps might be a much better way than speculation here.
* (Maybe obviously) Short/Long sleep studies with depression may be hopelessly confounded by common drugs (caffeine, nicotine, alcohol, benzos) which affect sleep a lot, and which depressed people would be prone to take, and unless you've looked into the methodology closely it might be worth batting an eye over (though maybe you did and you think it controls for this properly => ignore me, but it seems like the kind of thing that's hard to deconfound for many reasons, one way or another you end up with weird samples)
meant to say "electrode" not "neuron", but I can't edit comments :(
[warning: extremely speculative]
So we've established in this post that depression is caused by sleep, particularly REM sleep. It's well known that REM sleep is the only kind of sleep where dreams occur. When dreaming, your brain generates some arbitrary thing-that-resembles-a-narrative-story, and this thing further reinforces the intuitive idea of being a self-controlled sapient character in a grand narrative of the universe. In Buddhist/ish thinking, the idea of being a self-controlled sapient character is a myth and being enlightened causes one to intuitively grok it as a myth. So chaining the logic together we should conclude that enlightenment is a cure to depression.
I'm kind of stumped on the dreaming phase of your daisy chain. I'm predisposed to think of dreaming as the opposite of narrative reinforcement -- more destabilizing of one's habitual reality.
Have you polled enlightened Zen monks on how much they dream?
No, I have not polled enlightened Zen monks (or any people with a strong spiritual practice, for that matter) on how much they dream, and I'm most uncertain about the relationship between dreaming and narrative reinforcement as well.
Can anyone whose worked with or suffered from severe depression comment on the relative misery of depression versus sleep deprivation?
Not to be cliche but would you rather be depressed or sleep deprived?
I've had both sever insomnia and MDD, though whether it ever qualified as 'severe' depression depends on where you put the cutoff I suppose. In my personal experience sever chronic sleep deprivation feels almost identical to depression, with the exception that it is much easier to get rid of (just catch up on sleep). Acute sleep deprivation such as is used in the attempts to treat depression, is significantly better than the depression itself if and only if enough caffeine/other stimulants are involved that I can maintain some level of actual brain function.
Yeah, I think people are underestimating the kind of sleep deprivation involved to shift the depressive state. It's not "oh I feel really tired, I'd love a nap" levels, it's "Mr. Brain has left the building, I am starting to taste colours" levels.
I mentioned this in another comment above, but in college I found for a period that reaching a certain level of sleep deprivation would relieve me of a lot of my anxious and depressive thoughts. Of course this was sleep deprived to the point of functional impairment (body pain + inability of deep thought), which grew worse the longer I kept this level of sleep deprivation.
The short of it is I would prefer being functional and sleep-deprived to being functional and depressed, but I would prefer being functional and depressed to non-functional.
Never been formally diagnosed so take this with the appropriate scepticism, but that's a really tough question.
You can kind of get used to a functioning level of depression day-to-day, and it's only the really bad times that make it really bad. Being sleep-deprived to the manic state certainly lifts you out of the normal depressed state, but mania is its own problem. With me, walking into walls and bursting into inappropriate fits of giggling wouldn't be too good at work.
So I guess - for myself - I'd take the daily functional misery over the sleep-deprivation, but for the really bad times I'd take the mania instead.
Depression and anxiety: I find that if I am consistently getting less sleep (e.g an hour less a night for a week) I have increasing difficulty with basic mental tasks. Also the physical side effects of sleep depirivation of general bodily pains and getting sick more easily. Any uptick in mood is relatively minor and doesn't help much if I'm struggling at general life tasks
Its been years since I did the staying up all night thing, because it would mess up my sleep pattern for weeks afterwards, so wasn't really worth it
As others have already commented, sleep-deprived to the point of 'mania' is generally better than (deep/severe) depression. But, as someone that suffered from chronic sleep deprivation (that was 'environmental', e.g. due to a noisy sleep environment), I'd take everything short of suicidal depression than that. A night of no sleep, or a week or two of too-little sleep, can help depression, but anything longer or more severe than that can lead to psychosis, which can be much (much) more unpleasant.
Stupid thought on REM vs non-REM section: depressed people actually have MORE synapses, not less. Depression is a brain wave "brownout": barely enough to run everything at minimum capacity assuming nothing else taxes the system. As more synapses form throughout the day, the brain has to upgrade to full "blackouts" in various regions to keep the brain from permanently damaging itself, which frees brain waves to run everything else better, which is why they seem to get better as the day goes on, and why sleep dep seems to make things even better: more blackouts free up more brain waves for the rest of the brain. They get just enough non-REM sleep to normalize back to brownout levels, which starts the cycle over again. I would expect this to be falsified if the entire brain gets reduced function instead of just isolated regions during depression, and I would expect brain frying drugs like marijuana to help. I'm probably totally wrong though and welcome correction.
Back to reading
Pushing back on the "number of neurons doesn't change" claim, neurogenesis (new neurons being born and integrating into the circuit) is not only a thing (and despite some claims pretty likely to be happening in adult humans for a limited number of cell types) but is also proposed by some as the main mechanism for the positive effects of SSRIs, exercise, enrichment, etc on mood, since all of these increase both neurogenesis and mood.
Did you read the link? I thought the most recent research was that this didn't happen or is irrelevant - do you know of an update?
Some of you may be interested in the new paper from the Latham lab arguing that all synaptic modification is just part of Bayesian inference: https://www.nature.com/articles/s41593-021-00809-5
If seizures help treat depression, then why are people with epilepsy more likely to experience depression? Or do electroconvulsive seizures work differently than epileptic seizures?
Thanks for an interesting article 🙂
In my view to consider sleep deprivation as a valid approach is nothing short of insane. Reminds me of lobotomy. As a side note, the underlying specific assumptions of a computer-like functioning of the brain seem to me a strange caricature of current knowledge. I guess I hadn’t been connected to the community for quite long and forgot the strange belief system at work..
If rem sleep clears out the short-term memory and a having full short-term memory bank is linked to the reward system that would make sense from a more intense morning depression and lesser in the evening standpoint. Would then refilling of the short-term memory banks upon waking by reading something particularly interesting/enlightening (such as this blog) create a large number of new memories right off the bat and kick-start the day with a more elevated mood due to the well topped up short-term memory? Are there more effective ways of filling the short-term memory than reading, such as cramming Trivial Pursuit cards or listening to an audiobook on double speed (doing that feels like an information download straight to the brain)?
So the overall result is to increase "equality" of synapse strengths throughout the affected regions? This sounds kind of like destroying information, but maybe if the information is the pattern of connections making you depressed that's good?
From an information theory perspective, increasing equality doesn't necessarily reduce information. Suppose the brain had 1 really really strong synapse. That isn't much information. It isn't hard to point out which synapse is the strong one. On the other hand, if all the synapses were equally weighted, that carries no info. (But might make the info about where the synapses are easier to access.) Much more info can be stored by giving half the synapses a weight of 2, and the other half no weight. The distribution of synapse weights that maximises info (given fixed total synapse strength) will conceptually resemble the distribution of kinetic energy in an ideal gas.
The actual neuron weights will follow a geometric distribution (in this simplified model, where we are maximizing info with a bounded total weight and weight is just the count of some molecule), with a parameter that depends on the total weight bound.
TL;DR. A long tail of really strong synapses doesn't hold much info. A more even spread of strength can be more informative.
When I was in college, I was giddy-drunk if I went without sleep for a while, and that was fun. (Although I didn't drink in college, so I can't really compare.) I think my inhibitions just went way down and I'm normally too cautious.
Looks like they understand depression about as well as digestion. I can tell you for a fact that living with an optimist who loves you but won't put up with the attitude after a point can cure depressive tendencies over time.
>Is it fair to say that "decreased synaptic strength" is sort of the same thing as "abnormally low confidence on predictions"?
1. Consider that what may be involved is something like a "ready to hand" instrumental confidence in the effects of the {depressed, manic} person's actions; we can try to gloss this as corresponding to the epistemic confidence they have in their predictions, but that abstraction may be misleading in a lot of cases. (Famously it's hard to translate the ability to tie your shoes or execute a tennis serve into "predictions" or "decisions", and a lot of quotidian executive-function tasks are likely like that.)
2. What may be going on is that the depressed person's brain is *too good* at making the synaptic connections. If this manifests over the course of months, we call it a bipolar disorder. If their brain is somehow wise to what's going on and adjusts daily (nightly), then they wake up despondent/hopeless/directionless, go to bed slightly manic, and are at their most average functioning mid-afternoon.
3. This meshes with a lot of other pathways/covarying cognitive deficits. This kind of depression is connected to anxious behaviors like compulsive procrastination; that would make perfect sense if ingrained compulsive habits like e.g. checking blogs (and writing blog comments?) provide extremely predictable feedback. (This is something I think makes more sense on a "ready to hand" model than a literal "epistemic confidence" model; it's not clear why doing something just to be doing it would result from reliable predictions, since anxious procrastinators can usually predict when prodded that nothing valuable will result and they will regret procrastinating.)
4. It explains some stylized facts like: (a) why melancholy tends to attack people in relatively independent careers (they have more options and are vulnerable to decision fatigue early in the morning, and have fewer forced actions to get them through their low-functioning part of the day); (b) why they tend to have self-perpetuating circadian rhythm problems at night (they are most overconfident in the value of doing one more thing when they need to go to sleep, and have very firm convictions about tomorrow's agenda that they meticulously plan out when they're lying in bed); (c) why some people are convinced lifting weights helped their depression (very simple practical cause-and-effect relations, usually early in the morning); (d) why mastering some new hobby with quantifiable success/failure/improvement metrics often brings people out of a funk for a while.
I myself was in a melancholic funk a long time ago, and it stopped at the same time I started a ketogenic intermittent fasting diet that involved going to bed hungry and not eating until I finished my morning workout. At the time I thought that my rising spirits gave me the willpower to follow the diet, but now I wonder if it wasn't the opposite; starving my hyper-activated synapses in the evening forced me to bed and decreased the amount of overnight synapse-correction I needed. — Meanwhile I also wonder if the problem didn't originally arise because for a long time I tried to save time and diet by skipping breakfast, habituating my brain to absence of sugar in the morning. (Consuming my morning caffeine with artificial sugar instead of real sugar probably didn't help.) — That said, I wasn't at my most productive while practicing intermittent fasting; my most productive years in recent memory were again a period where I woke up in a fog, got nearly everything done 1pm-5pm, sometimes felt like trying to get a lot done at midnight, and sometimes had sleepless nights. So it's possible the overall pattern of strengthening synaptic connections over the course of the day isn't purely dysfunctional or diseased, even if it carries risks.
As someone with lifelong depression, I can attest that the sense of being "fully activated" only late at night is very real. I do my best creative work between about 10pm and 2am. I used to intentionally plan all-nighters when I had difficult school work or papers to complete; I eventually just realized that if I wanted 100% productivity, I had to do it late at night.
At the ends of semesters, I'd sometimes take a 36 or 48-hour stretch of final paper writing during which my productivity would reach *extreme* heights. For example, producing multiple graduate-level papers which received high grades in just a few days - something I *cannot* do normally. Toward the end of these periods, I'd experience a sensation I'd describe as feeling "too awake" - extremely alert, quick thinking, rapid ideation, borderline manic. If you've ever taken ADHD medications in a somewhat too high dose, it was a very similar feeling.
So, this discussion seems extremely accurate to my personal experience.
" I'd experience a sensation I'd describe as feeling "too awake" - extremely alert, quick thinking, rapid ideation, borderline manic"
Yeah, I've had similar feelings the few times I've been very sleep-deprived (no or very poor sleep). That's why I'm a bit dubious about "keep the depressives awake and it will reset them" because 48 hours with no or little sleep will make anyone manic.
Also the fun sensations of "tons of energy, mind racing, and walking into walls, doors and anything in the vicinity because for some reason my legs won't work on straight lines anymore" after 36-48 hours wakefulness 😁
So, I used to do this in college as well. I only stopped because the "hangover" from doing this kind of thing ended up being more disruptive than the benefits I was getting.
I 100% can program better at 3 a.m. then I could at midnight which was, in turn, better than I could program during the previous afternoon. The commenting gets way worse, but the solutions are much better. This is useful to me, and I'd love to get to use it more often.
The problem is, the effects of doing this kind of thing just last forever. A 36 hour period of wakefulness has me feeling like crap for around 1-2 days. Anything over 48 hours give me cold-like symptoms for up to a week. Is there any way to get the benefits of staying awake without the aftereffect?
Is it possible to self-medicate depression by restricting sleep or reorganizing your life around it somehow? I'm thinking about shifting your work and important stuff to the evenings/nights and instead taking it easy in the mornings. I was reminded about how as a student I tended to gravitate towards doing more studying in the evenings (except for bar nights I suppose). It seemed to work then, and aside for a preference for having evenings off I don't see why it wouldn't work now as well.
Alcohol, in high doses, decreases REM sleep: https://onlinelibrary.wiley.com/doi/full/10.1111/acer.12006
This would seem to support a more complex view of the role of REM sleep in depression. SSRIs reduce REM sleep and fight depression; alcohol reduces REM sleep and worsens it.
Throwing another anecdotal evidence on the pile, I've always been a night owl, I find mornings difficult even though I've had to get up early for years, and on top of that I tend to have insomnia - either it takes me forever to fall asleep, or when I do fall asleep I keep waking up throughout the night.
And then when I get the opportunity to catch up on sleep, I can sleep for hours upon hours. I don't know how much linked that is with depression, but it certainly *feels* like it.
The times I have been sleep-deprived, I have ended up bouncing off the walls, but I don't think it had much of an effect on my depression; instead of feeling unmotivated and 'I hate myself', it was "Whee! I can finish all these tasks! *And* I still hate myself! Oh look all the straight lines are all bendy..."
What of other animals? Bats sleep much more than people, but have much simpler brains. Wouldn't you expect the much greater number of synapses to lead to some notable difference in sleep patterns or the types of chemicals released in sleep or something?
At least in children, "wake up grumpy but gradually cheer up" seems to be a lot more common than the reverse.
I was also thinking that dementia stems from a sleep disorder. It’s just that my mother who had something going on (neurologist could not define it other than dementia) would have moments of clarity then drift off to what seemed like she was describing a dream when she spoke. Meaning everything she was saying was disconnected like what happens in a dream.
What does this imply about people who need way less sleep than the average person?
Back in college the seniors would have a rotating shift of "night watchmen" who had to wake up and patrol the dorm a couple times a night to make sure nothing crazy was going on. When it was my turn I remember be constantly unnerved that the same 5 or 6 people out of a thousand and change were always up and about at 3 a.m.
Could it be that the "normalization waves" of these folks are abnormally strong and accomplish the task in just a couple hours? Or maybe they are susceptible to depression effects and have just evolved a strategy of exposing themselves less to sleep to maintain better mental health?
Glymphatics anyone? Searched for this in the comments along with lymphatics and aquaporin and go not hits.
Recently(ish) the extracellular mass transport of fluid has been shown to double during sleep. This is thought to be protective for Alzheimer's disease for example. It is also highly dependent on aquaporin 4 expression.
I note that early on Scott writes "As if depression is caused by some injury during sleep". The opposite could be true.
Hypothesis: some semi-soluble compound in the extracellular space is beneficial to depression but sleeping allows it to be washed away. During the day it reaccumulated.
So this coincides with a weird issue I'm investigating in my own brain.
For background, I have atypical depression (diagnosed, used to be medicated, mostly in remission for now) and probably ADHD (diagnosed as something ADHD-adjacent as child when that was not a category in my country, unmedicated). Mornings are universally the worst part of the day, both in terms of mood and sleep latency. Coffee helps a little bit but is mostly drank for pleasure. Brain fog is present during the entire day, but I manage.
The thing I discovered by accident is that drinking coffee in the evening gives me superpowers. I never liked to work in the evening, instead reserving it for chilling out, but with a sufficient burst of caffeine I get an incredible amount of stuff done, concentrating effortlessly while in a hypomanic state. Distractions that normally completely destroy my productivity are no problem - I can easily sustain conversation via IMs and not lose track of work, for example. This effect works _only_ in the evening and late afternoon.
This makes absolutely no sense under the standard dopaminergic (no relation to circadian rhythm) / adenosine antagonist (related to circadian rhythm, but it just makes you feel non-sleepy) explanation of caffeine action. The circadian depression voodoo explanation is... tempting, to say the least.
"This sounds kind of like destroying information, but maybe if the information is the pattern of connections making you depressed that's good?"
If I've got a textbook in mostly nine-point type, but one random sentence on each page is in forty-point bold, I don't lose any information and I do gain more efficient access to information if I reformat those sentences to the same font as the rest.
If it's not random sentences but the most recent sentences (the stuff I experienced yesterday and was supposed to reformat down to standard-archival-format), that sounds like it might combine synergistically combine with some poorly-chosen trapped priors. If my recent experiences are of the "life sucks, there is no hope" variety, and those keep getting overweighted, then I won't be able to draw effectively on my previous experience of life not sucking.
In which case, excessive sleep alone won't always cause depression, but would aggravate other potential causes of depression and impede natural recovery.
What about a simple alternative explanation: patients are going through withdrawal in the morning, because they haven't taken their SSRIs or other meds in 8+ hours.
Huh! I've noticed that during depressive episodes, my dream content "lags" by a few days. For example, if I watch a movie, those characters will show up in my dreams three days later, where normally they'd show up the same night. Related, maybe?
I have an interesting perspective of this, having done night work for over 30 years. Not just 30 years but for much of it six nights a week. Anyone who has worked the third shift can relate to the unnatural sleeping patterns. My day off (Friday--today in fact) I spend up to enjoy life the way day-trippers do. I am just about at 24hrs and going strong. Now I would not want to take a long drive at the moment, but lets hope this has some clarity.
Regarding depression, my own personal experience has been that sleep, deep sleep, is much needed to ameliorate it. Now, there is merit in sleep deprivation. The longest run I've had is three days straight. I fainted eventually (at the breakfast table!) but here's the odd thing--on the way home, I felt like a million bucks. A few things played into this I believe. The weather was warm and sunny, I kept myself moving (stopping is the bane of the sleep deprived) and was relatively trouble free. A miserable, rainy day in the winter after my cat just died might have another outcome. Never-the-less, I still fainted. My body pulled the plug on my mind.
Over the years, I have noticed a curious pattern. While I can stay up for long periods of time, I find after deep sleep, my mind fires on all cylinders. Let me tell you about my old friend Frank. His father was in a rest home and I would visit with him from time to time. Frank would warn me his pop would need a good 30 minutes to get "his brain moving" as he described it. He said it was almost like the blood had to get to his brain and took that long. Slowly his pop would come out of the malaise he was in, know our names, engage in conversation. I said I often stay up all day Friday. Depression (when it hits me) is Saturday morning. Early. 5am maybe after a good 5 hour sleep perhaps. I could lie there turning over thoughts that somehow seem the have the better of me. So, I one morning thought of Frank's pop. Thought perhaps my own blood had to get flowing, and simply chose to let my mind worry later, at a time when my mind was better apt to take the trouble on. Just as I would not fancy a long drive now for safety issues, I applied the same to my mind in that state. Fast forward to Monday night now. I am driving in, bright and alert, same troubles and "I've got this', I can say to myself. All my mind needed was rest. I've become a big advocate of proper sleep to bring clarity and convinced lack of sleep causes shotty thinking. I don't deny the research here, to some degree have experienced it, but also the benefits of a rested mind to achieve the same. I've seen the depression "reset" Saturday morning but the exact opposite in direct proportion Monday night sleeping sound all day.
Tim Shaw
I've often felt like i get more tired and depressed towards the end of the day. I have a demanding job, two kids, and aging parents. Towards the end of the day, I often feel down about almost everything. That seems to be the opposite of what the article describes.
When i wake up, this generally isn't the case. I've often used the image of something like 'light' that illuminates the space of possibility nearby - both what's happening now, and what might happen. When I feel good, this light is bright, and when i don't feel good, it's as if the light is 'dim' - everything just seems more dismal and hopeless.
That might make sense of connection strength gets stronger over the course of the day, and strong connections require more energy to maintain. If the connection strengths all get downgraded evenly over the course of sleep, then it takes less energy for my brain to map out all the various connections between things in my immediate environment and, as a result, i can more easily perceive the subtle, positive things that make life seem better.
It often feels, when I'm tired or angry or sad, like my experience of the world reverts into a 'cartoon' state, with all of the unpleasant feelings highlighted. That might seem consistent with only the strongest connections getting enough energy to run?
If we follow this theory that (certain patterns of) increased synaptic connection treats depression, and evidently our activities over a day tend to produce such, do we know if there are particular activities that do so disproportionately? It might make sense to front-load those.
There are some candidates on how top-down theories like predictive processing and bottom-up theories like homeostatic disregulation can meet.
In predictive processing, there are two major candidates on how/where the downwards prediction and the upward sensory information might actually be implemented.
One is that different parts of the layered structure (https://en.wikipedia.org/wiki/Cerebral_cortex#Layers_of_neocortex) of the cortex are used. Details are complicated, but probably upwards stream mostly arrive in layer IV, while the comparison with prediction might be done in layer V, or possibly in layer II/III. We don't understand the details of this "canconical microcircuit" very well yet, but there is a lot of structure.
The other (complementary) candidate is that different neurotransmitter are involved. Upstream information uses mostly AMPA (which is a LOT faster and is responsible for the lion share of energy consumption), while predictions probably/perhaps involve a lot more NMDA receptors (slower and longer-lasting, strongly involved in plasticity and learning).
Homeostasis does not affect all these structures the same way. As far as I understand Tononi's theories, mostly the AMPA part is upregulated over day and downregulated at night. Also, different layers are affected differently.
Once you know such things, you can start to speculate. If AMPA is stronger in the evening, does that mean that predictions are relatively weakened compared to sensory information? I don't think it gives a coherent picture (yet), and I don't think that we are at a point where we can match top-down and bottom-up theories. Probably we won't be there in the next 10 years. But I don't think it is hopeless in the long run.
Maybe but I still out substantial credence on finding out the sleep result reflects affects to the endogenous opiate system/cycle. I'd love an experiment that measured levels of those endogenous opiates over the time period.
What about schizophrenia? Sleep deprivation often leads to psychosis and the negative symptoms are a lot like depression symptoms? What's the interaction there?
It's weird to me to hear that sleep deprivation can help with depression, because I'm pretty sure sleep deprivation makes me *more* depressed, at least in the long term. But possibly short-term sleep deprivation is helpful and long-term sleep deprivation is harmful?
I'm wondering if what you wrote here can be used to explain why SSRIs take a week or two before they start working.
I really don't appreciate that you used a Sci-Hub link without warning
On the contrary, I think this is great. Thanks Scott for using Sci-Hub (you could add a warning if you want).
During college, I did a research paper on synaptic scaling during NREM sleep, and ran into Tononi on a ludicrous number of articles. I started noticing his name popping up on all the interesting ones, so I just did a search by author. I don't know how the **** he does it, but his name is on all sorts of crazy papers that are clearly trying to become a groundbreaking paradigm shift in their own way. At this point I'm convinced that he has a network of contacts across the entire field that give him early-warning alerts on weird new papers so he can swoop in and coauthor them.
For what it's worth, btw, the consensus among researchers ca. 2018 was shifting towards Tononi's NREM model being less helpful than the other big model in the field. There were a couple of studies that looked really slam dunk until one of the big names on the other side pointed out that they were anesthetizing their mice with an agent that specifically interfered with the part of the brain they were looking at. I don't know what the fallout was from that or if Tononi's people recovered, but it was an oof moment for sure.
Huh that explains it for me, thanks! "It" being: I sometimes feel depressed early in the morning, like first 5-30 minutes after waking up, although I'm normally not and never been diagnosed. It happens without any apparent reason like sleep deprivation, hangover or shitty day ahead/behind. And it's not just low energy, I literally feel like I'm worthless, my life sucks, things I said and done yesterday are all failures etc, even though I remember it's not what I was thinking yesterday and know I'll probably feel differently in half an hour. I eventually assumed it's some sort of a brain glitch and just made a habit of not thinking of any such matters right after waking up, now I finally know what kind of glitch this (probably) is.
Double curse for people with epilepsy, who have to maintain a rigid sleep schedule, whether they like it or not, to ward off seizures.
As someone who experiences depression, it personally feels more like “your emotional system only has half as much RAM as usual today, so you're not going to be able to trust anything you think”. I don’t feel sad per say, I just don’t feel good either. Whenever someone asks me what I would like to do/eat/see/whatever I say “I don’t know” because none of the options seem like they would be better than the others. I feel like maybe I would be better off if I were able to at least keep strong negative associations with things in mind, because then I could figure out what I want though process of elimination.
I know that sleep lowers seizure threshold / increases likelihood of seizures. seems relevant here
Staying up for ages takes you out of yourself. Same as exercise does.
In case anyone still reads these comments, since I read this article when it first came out I've started viewing nights where I have to sleep less as medicinal. "Okay, I can only sleep 5 hours tonight. That's good! I've been sleeping too much and feeling sad. Maybe tomorrow I'll feel better. :)" It might be a placebo, but so far this mindset has made me 1. more of a morning person. 2. happier. :)
I have always wished I can offer myself for any well funded research into the connection between sleep and low grade chronic depression. I have always believed and told people that I feel there's a strong link between the two phenomena.
I sleep a lot so much so that I was nicknamed Jonah the sleeper in childhood. I once went to my university health center to complain about sleeping too much. I meet many of the criteria for Narcolepsy and all the criteria for hypersomnolence. I once took modafinil (which I first heard about on this blog) and it worked tremendously which made me stop for fear of dependence. But it didn't relieve my depression, only excessive sleepiness.
Much of what this article says about the complicated relationship between sleep and depression applies to me. I have long discovered that I experience a negative rebound from the kind of overactivity associated with having fun such that I almost always have a mood crash afterwards but I have tended (obviously wrongly) to interpret it in psychological terms: I feel guilty after having so much meaningless fun such that I feel the need to atone by returning to my true original self. Alas, if this article is anything to go by, it is just pure circadian chemistry!(?)