My Secret To Retiring Early With Only A $4 Million Net Worth And Two Kids

The following is a guest post from a long-time Financial Samurai reader named Joona. He shares his secret to retiring at age 41 with a $4 million net worth, a wife, and two kids ages five and eight.

I really love hearing about early retirement case studies and how my writing has helped people reach their financial goals. After all, I’ve been writing about my own case study of achieving financial independence since 2009. It’s been a fun journey full of twists and turns.

Previously, I had written about how retiring early with $5 million is extremely difficult for families in expensive coastal cities. Therefore, reading this case study on retiring early with $1 million less is particularly insightful. Take it away Joona!

Retiring Young With $4 Million And Two Kids

There’s an old saying, “Even if you win the rat race, you’re still a rat.” When I first heard the saying, I was offended. I had just finished going to law school for three years and had $100,000 in student debt.

But after five years of working as an associate, I recognized the truth in the saying. The hours were long, but I wanted to make partner by my 35th birthday. I didn’t. I also wanted to have my first child at age 35, but that didn’t happen either.

After 11 years of grinding it out as an associate, I finally made partner at age 37 in 2019. Coincidentally, I also had my first boy at 37 as well. I thought my life would be much better with more pay, less grunt work, and more schmoozing.

But being a dad changed my perspective.

Instead of looking forward to going out for expensive steak dinners with the finest bottles of wine until 10 pm, I felt guilty leaving my wife alone to take care of our son. Instead of spending five hours on the golf course with prospective clients, I longed to see my son’s first milestones.

The pressure to be both a present father and a great lawyer was making me miserable. Something hand to change.

How Much Money I Made As A Lawyer

For those curious, here’s how much I made as a lawyer. I didn’t work for a big law firm like Cravath, Wachtell, or Sullivan & Cromwell. Instead, I worked at a boutique law firm that had more humane hours but paid less. It was the tradeoff I was willing to make.

Year 1 after law school at age 26 – $100,000

Year 2 – $110,000

Year 3 – $130,000

Year 4 – $145,000

Year 5 – $160,000

Year 6 – $170,000

Year 7 – $170,000

Year 8 – $180,000

Year 9 – $190,000

Year 10 – $200,000

Year 11 – $250,000

Year 12 – $275,000

Year 13 – $340,000

Year 14 – $430,000

Total earnings after 14 years: $2,850,000

The amounts may sound like a lot, however, I went to law school for three years and took on debt. Further, today’s starting salary for 1st year Big Law associates is around $200,000 plus a $25,000 stub bonus.

Maybe I could have made a million dollars a year after 20 years. But I wasn’t going to stick around that long to find out.

My saving rate averaged about 55% for my entire career. During the last two years of work, I saved 75% of my after-tax income. When you have a goal to escape, saving money because extremely easy.

Net Worth Progression As A Lawyer

Here’s my rough net worth progression as a lawyer. By the end of year four, I had paid off my law school debt. I also didn’t have any undergraduate school debt because my parents paid for it.

70% of my savings went towards three Vanguard index ETFs. The other 30% of my savings went toward buying a multifamily property for rental income.

Year 1 after law school at age 26: -$10,000

Year 2: +$20,000

Year 3: +$50,000

Year 4: +$120,000 (paid off law school debt)

Year 5: $200,000

Year 6: $300,000

Year 7: $550,000

Year 8: $850,000

Year 9: $1,030,000

Year 10: $1,350,000 (bought multifamily property)

Year 11: $1,620,000

Year 12: $1,900,000

Year 13: $2,200,000

Year 14: $2,500,000 (today)

With an estimated $2,500,000 net worth, I decided to call it quits. Being a partner at my law firm for three years was enough for me to leave with no regrets. I reached the pinnacle of my profession and the only thing I would leave behind would be money.

What I’d gain would be more free time with our now two children. They are ages eight and six. The one thing about having kids is that it makes time go by faster because they are changing so quickly. Before I know it, they’ll be heading off the college.

Due to inflation, a $2,500,000 net worth isn’t what it once was. Sam has written that to be a real millionaire, you need at least a $3,000,000 net worth. But for me, it was good enough to say goodbye, so I did.

The Secret To Retiring On Only $4 Million With Kids

Here’s the thing. I didn’t just retire early with a $2.5 million net worth. After all, there have been two income earners over the past 14 years. My wife, whom I met in law school, is also a lawyer and has no plans of retiring any time soon.

Ever since we first met at the campus cafeteria, we’ve been on the same page about our careers and financial goals. It’s one of the reasons why we get along so well.

She was two years behind me in school and still has the same fire for being a lawyer as when I first met her as a third-year student. In comparison, I was starting to burn out after my 11th year. She’s up for partner this year and plans to work until she’s 60.

My Independent Wife

Since we first met, my wife has always been an independent woman who has sought to build her own wealth. She’s also been extremely into finance since high school. Neither of her parents had job stability, which often made her worry. Partly as a result, we’ve always had separate banking accounts.

Here’s another life hack. We also aren’t technically married, which saves us between $15,000 – $25,000 a year in marriage penalty tax. We realized we didn’t need a certificate to prove our love for each other. As a result, we’ve saved over $250,000 in marriage penalty taxes during our relationship.

My wife wanted to see how much wealth she could accumulate on her own. And like me, she didn’t understand why one plus one doesn’t equal two in the government’s eyes. As a high-income earning woman, she feels the government is wrong to penalize women for wanting well-paying careers.

Over the past 12 years of her career, she has been able to accumulate a net worth of about $1.5 million. In other words, our combined net worth is about $4.0 million.

Related: The Average Net Worth For The Above Average Married Couple

Is A $4 Million Net Worth Enough To Retire Early?

We have about $2.8 million in investable assets generating anywhere between $60,000 – $120,000 a year in passive investment income. The income range depends on capital distributions from various private real estate funds we own.

On the low end, can I live off $60,000 a year in gross investment income or about $50,000 a year in net investment income? I can. But I don’t.

We reinvest 100% of our investment income and capital distributions every year in order to generate even more passive investment income.

Given I’m only 41, I want to keep building our investment income so that by the time my wife retires in 20 years, we’ll really be set.

So how do I survive and take care of the kids?

Well, that’s easy. My wife makes about $350,000 a year as a lawyer. If she makes partner, she will likely make $500,000 or more in the next several years. Even if she doesn’t get promoted, she will likely continue to make $350,000 – $400,000 for the foreseeable future.

With a $350,000 gross income, after taxes, she brings in about $270,000. Given our budget is only about $160,000 a year, we’re left with about $110,000 to invest in stocks, real estate, bonds, private funds, and to save for our children’s education.

But aren’t you just a stay at home dad and not retired?

Absolutely. I’m a stay at home dad who happens to have worked for 14 years and amassed a $2.5 million net worth on my own.

I no longer work because I don’t need to. Instead, I want to decompress and spend more time with my children before they leave the house.

$160,000 Spending Budget Breakdown

Here’s a rough estimate of our $160,000 a year budget. We’ve been spending about $160,000 for the past two years so we feel confident that we’ll be able to maintain this budget for four people going forward.

I tried to be as concise as possible. In the past, when I would do our budget, I discovered we actually spent way more than we thought. With this budget, I think we could easily cut $20,000 a year if we need to.

Based on the figures we live a pretty normal middle-class lifestyle.

We send both our children to public school and live in a comfortable three bedroom, two bathroom home with a backyard and an office. Our house is about 2,200 square feet. We’re also saving $20,000 a year for college in a 529 savings plan.

After sixteen years, our youngest will be done with college and our expenses will likely drop. However, we’re still budgeting in some expenses to help our kids after college just in case they have a difficult time launching on their own.

Spending $160,000 a year requires earning a gross W-2 income of about $213,000 based on a 25% effective tax rate. Therefore, our target passive income target by the time my wife retires in 20 years is about $220,000, inflation adjusted.

Early Retirement Is Easier With A Working Spouse

I’m very proud of my wife’s career aspirations. Unlike me, who is completely burned out from the corporate world, she’s still highly motivated. But given she is two years younger than me, maybe she’ll feel differently after a couple of years as a partner.

Saving and investing aggressively for 14 years probably makes up for 70% of the reason why I was able to retire at 41. 30% of the reason is due to my wife. I could happily live off $2.5 million as a bachelor if it was just me and one child.

The balance in our household is good as I take our kids to and from school. Further, I pay all the bills, order or prepare most of our meals, and keep up the house in order. I’m also the one who arranges all the repairs and maintains our rental property.

The Rise Of The Female Breadwinner

For many households, the situation is the opposite. Men are often the ones who need to be the breadwinners while their wives stay at home to take care of the children and household.

Thankfully, in our increasingly equal society, more mothers are becoming breadwinners. They should be as the majority of college-educated adults are women.

Spending as many years possible working after spending so many years in college and graduate school is the rational thing to do. For me, 14 years of work after law school was enough.

I hope our daughter can one day find a career she loves and make a great living as well. I’m proud my wife is setting a great example for her. If her husband decides to stay at home and raise the kids, I might just help out too!

At the end of the day, my wife trusts nobody with the kids more than me. It gives her a lot of peace of mind that I’m the one taking care of them while she is working. When she was growing up, her parents were always working.

The Confidence To Retire Early With A Working Spouse

I will admit I wouldn’t have felt comfortable retiring early if my wife also wanted to retire early with me. $60,000 to occasionally $120,000 in gross passive income is not enough to fund our $160,000 a year ideal budget. We’d also have to pay over $2,000 a month for health insurance if we didn’t have subsidized healthcare.

If my wife wanted to join me in early retirement, I think we’d need closer to a $7 million net worth. With around $6 million in investments, we should be able to generate over $200,000 a year in gross passive income to fund our lifestyles.

If my wife works for another six-to-eight years and if our investments cooperate, we have a great chance of getting there. And by then, my wife will still be in her 40s to enjoy the early retirement life with me if she so chooses.

It’s true what Sam says. Your financial independence number is not really real if you don’t change your lifestyle. In my case, having a working wife has given me tremendous courage to leave my law career behind.

Consistent Encouragement Is Needed

I’m only eight months into my early retirement lifestyle so take my perspective for what it is. But so far, I’m enjoying the much slower pace of life. One surprise I discovered is my chronic interior elbow pain finally went away after two years. If I knew my anxiety and chronic pain would have gone away, I would have retired a year earlier.

Sending my kids to school and picking them up to go to soccer practice and other events gives me purpose from Monday through Friday. So does making sure all our household items are in order.

I go for bike rides and hikes every day, which have improved my mood and overall happiness. I’m also spending more time looking for investment opportunities.

So far, my wife likes the balance. Me not working has reduced her mental load so she can spend more time focusing on work while at work. When she comes home, she loves being able to unwind, which was harder while I was working.

I’ve done my best to follow the tips of other men who also retired early and have working spouses. For example, I’m always encouraging before she starts her workday. I’m also always there whenever she needs to vent about work issues. And if she needs me to run an errand, I’ll do it no problem.

The more I can anticipate her needs before they come up, the better.

Surprisingly, I don’t miss the money. The freedom I’ve gained from early retirement has more than made up for the lost income. Given I saved and invested most of my money anyway, earning more money no longer improves our lifestyle.

Manage Your Happiness In Retirement

Finally, if you plan to follow my lead, I recommend being careful about showing too much outward joy as an early retiree, especially on your spouse’s bad days. Instead, try to mirror your partner’s mood so any resentment about your new lifestyle never grows.

At the end of the day, you want both partners to continue living their desired lifestyles. Over time, views about work and retirement will change though. So stay flexible.

During the tougher days, we remind ourselves that we’re doing what’s best for our children. In addition, if we ever need to earn more money, I can always do some part-time consulting or go back to work.

But for now, both of us trying to make the most money possible doesn’t make sense. I’m not ready to go into decumulation mode yet. But I am happy to step away from the grind.

When the time comes for my wife to retire, I’ll be there waiting to show her around.

Readers, what do you think about retiring with a family on $4 million? Do you think you could do it? Are you noticing more men retire early while their wives work?

For more nuanced personal finance content, join 50,000+ others and sign up for the free Financial Samurai newsletter. To get my posts in your inbox as soon as they are published, sign up here.