Mortgage Secrets Podcast: How Much to Put Down

Business adviser analyzing financial figures denoting the progress in the work of the company

MORTGAGE SECRETS: EPISODE 3

How Much to Put Down

One of the biggest conversations you will have about buying a home is around how much money you should or shouldn’t put down on your home loan. Putting a chunk of money down on a mortgage is a pretty impactful decision, of course. Once you sign the dotted line and hand over the check, the money you put down is no longer easy to access.

If you’re thinking about buying a house, this episode of Mortgage Secrets will clear up some of that confusion. “Limiting factors” may seem a daunting phrase, but the secret to finding what works for you is exploring where you are financially and what options you have available. Sharing his knowledge from 17 years of experience in the mortgage industry, John Downs of the Downs Mortgage Group explains how you can figure out how much you should put down and where your money should come from.

This is Mortgage Secrets Episode 3

This is Mortgage Secrets Episode #3, and I’m John Downs from The Downs Morgage Group @ Vellum Mortgage. By now, you’re thinking about buying a house, and you’re trying to figure out:

How much money do I need?
How do I get it? Where is it coming from?
Does it need to be a lot or a little?

The biggest conversation—and the part where we slow down the most—is this very topic.

How much should you put down, and where should it come from?

In mortgage lending, you’ll hear me talk about limiting factors all the time. What are you limited by? Are you limited by how much money you make? Are you limited by the debt that you have? The credit you have? Are you limited by your own self? Are you limited by cash?

First, we have to figure out what you have. What do you have access to? Then, based on that, what are your options? Again, it’s just a recurring theme you’ll always hear. Where are you? What do you have? What are your goals? What does it look like today with all that in mind? On the cash piece, we typically start with that. “Just tell me what you have. Where are you today?” Then we can start digging into the possibilities.

I’ve met people from all walks of life in every single financial situation or picture you can possibly draw. I would say that cash tends to be the biggest limiting factor for most people. Incomes seem to be good, and you can have your debt under control, but depending on what market you live in (and, of course, what all the studies say is that) cash tends to be the biggest limiting factor for people.

Let’s say you are one of those people for whom cash is not really your limiting factor, and you’re just trying to figure out what you should do.

Again, I’ll use a very specific phone call that’s top of mind because it happened yesterday, but it happens all the time. Throughout this conversation, it was, “John, I have up to $120,000 to put down,” and we enter into a conversation of, “Well, is it the best thing to put it all down?” It started with, “I think I want to put it all down—what can I buy with this?” and then it morphed into a conversation of, “Well, what if you could do it with less? Is that the right thing to do? What are your investment patterns and goals?” Our normal Downs Group conversation.

It ended with us agreeing to show a ladder of different down payments—5%, 10%, 15%, 20% down, with monthly PMI, without—to really dive into all of those in order to unpack and figure out what the best path was for them.

There are a lot of reasons that putting less money down makes more sense, and it’s not that our mission is to try to have people finance the absolute most that they can. It is, again, just to reinforce the idea that every dollar you have can be earning or saving something. Where is that money? What is that money doing? If you didn’t put it down on this house, what could it be doing? What is the savings if you put the extra money down? Do you have enough money if something happens—something breaks, your car dies, a sudden whatever that needs cash? All those what-ifs of life, the things that can be somewhat predictable, and the things that need to be discussed before you put that money down.

One more thing with lending…

Once you put that big down payment, it’s kind of down. You’ve got that mortgage. If you want that cash back out, it’s not as easy. Sometimes you need to look at second mortgages, and maybe you refinance, but what if rates are a lot higher? There should be a lot of thought because you can always put more money down in the future and change your payment (the industry term for that is called a “recast”), but you can’t really pull that money out as easily.

I get a question a lot of “Well, what’s everyone else doing? What should I do?”

In my market, I’ve got to tell you—it’s all over the place. There is a lot of family gifting right now, strategically passing money downstream. I would say most are looking at the lower down payments. If you kind of look at today’s homebuyer—50% to 60% are millennial, super young. If you’re young, you’re just starting life; you’re just making money, you probably don’t have buckets of cash just sitting there. Routinely, I think the lower down payment is the one that we see day in and day out, and that we have the most conversation about.

Now, I know that a lot of people would say that if you don’t put a lot of money down, it’s risky. I don’t quite think that’s true.

It’s a weird story to talk about China while we’re talking about putting a down payment on a house, but when they want to slow their economy, do you know what they do? They force bigger down payments. They wake up one day, and they say, “Hey, now everyone needs to put another 10% down.”

The U.S. kind of did that by themselves. The private markets did that in 2007, 2008, and 2009. Everyone needed these big down payments. But I learned a lot of lessons back then. I learned what loans performed and which ones didn’t, and what we found is that when income was fully documented, regardless of what happened in the market, the loans performed really, really well until the defaults became more strategic, which is a whole different conversation.

But in today’s world, once Wall Street and investors and lenders get confidence in housing—where they can look out 5 to 10 years and see steady appreciation somewhat all over the place—if they think properties are going to appreciate 2% to 3% a year, they know that with a principal and interest loan, you’re paying your mortgage down about 1% to 1.5% a year out of the gate. So, every year, you’re building 5% equity.

Lenders tend to just look at this and say, “Well, if I let you put 5% down today, I will have my 20% equity in 3 years.” What’s going to happen between now and 3 years? What is the economy going to do? What do I think this person is going to do? Do I think they can do well between now and then? That’s really how lenders are looking at this situation of low down payments.

Some people out there say, “Well, I kind of feel like a scumbag if I don’t put a bunch of money down.” I just disagree with that. I think there are loans out there that enable and foster home ownership, and if they exist, the payments fit, everything makes sense, and your goals align, you should do it.

In today’s market, there are actually quite a few low down payment options for people to take advantage of.

Again, markets always evolve, and I think, right now, we’re in the cycle where things continue to get a little easier. Just in the last year or so, both Fannie Mae and Freddie Mac came out with their low down payment program. They’re kind of rivaling FHA. FHA says 3.5% down. Fannie Mae and Freddie Mac now offer 3% down. For those that live in a high-cost market—think big-city and expensive—you can now do 5% down loans all the way up to, in some markets, $660,000. You kind of bring in the second mortgage—which, combination structures are also something that’s used quite frequently for low down payments—and you can see people put 5% down and still buy up to a $1 million house. It’s out there, and they’re actually priced pretty well.

Now let’s talk about where the cash could come from.

Whenever I ask people, “Give me a big-picture overview of where all of your money is,” usually, it starts with, “My checking and my savings.” Then it maybe goes to some mutual fund they have, and then it spills over into some rollover IRAs or 401Ks or something, and then the 401K conversation. And then, if they’re blessed, a gift from Mom, Dad, Grandma, or Granddad. Those are the prevailing “Where does money sit?” Well, that’s where you can get it.

And a question I get a lot is, “What should I do with my 401K?” and that question goes both ways. It can be, “I know I have it, but don’t even ask me to use it,” or it can be, “I have it, and I’m going to pull out the max.” They’re already predetermined to “I am taking out this money.”

I have a pretty simple idea about 401K and taking money. Rule #1: Don’t ever, ever liquidate it. Don’t take a distribution where you’re actually going to have to pay taxes and pay a penalty because you can never put that money back in, and the same goes for IRAs. I get this one a lot: “Hey, I’ve got this Roth. It’s $20,000 from an old rollover. I can pull that out, and I don’t have to pay any penalties or taxes,” but those contributions and 401Ks and IRAs, you have a finite amount of money you can put into those vehicles per year, and once you pull that money out, you can’t go put it back. Just let that money grow forever. That’s the idea behind IRAs, Roth IRAs, and 401Ks. Just let it stay in there.

Now, to use your 401K, you can, in most cases take a loan. I believe that your 401K is invested in something and is designed to help you build wealth. If pulling some of that money out and putting it into a real estate asset that aids in you building more wealth in the future, it’s not a bad idea. Now, the problem is just knowing the market and how long that money is extracted. In other situations, we would talk about how markets evolve, and putting in over a long, long, long, long time is the key to building lots of wealth.

If you’re going to pull that money out of the 401K, understand where the market is. For instance, right now in 2017, we’re skirting along all-time highs. It’s probably a pretty good time to take some money out of a 401K. 2009? Not so much. If you pulled the money out in 2009, market rally is 100% in a couple years, you lost big upside.

My personal rule is:

How much are you taking?
How fast can you pay it back?
Do other loans exist that allow you to put less money down, so you don’t even need to take it?

I think that’s the important statement to take away here with 401K loans.

So, you’re the kind of person where you maybe don’t have enough vested money in your 401K to take it out, or maybe you’re planning on changing jobs, which really sort of messes up the payback schedule, yet you’re still trying to figure out a way to put money together.

The thing that we’re seeing a lot of right now is family gifting.

I really wish that families would sit down and talk about this topic deeper because what I find is that most of the kids are afraid to go to Mom, Dad, Granddad, or Grandmom and saying, “Hey, I’m thinking about buying a house, but I need a little bit of money. The older you get, generally speaking, the investments that you’re in are less and less risky, and you might just even be holding cash. There is a strategic reason for families to take money and pass it on downstream to then give it in the hands of someone else, so they can invest it for the next 40 or 50 years.

What I’ve found is when that conversation happens, and it’s well thought out, and the budget is put together well, and it fits where you are in life, most family members are very open to it and, quite honestly, pretty excited to help out. I think that’s the biggest place to turn. Just look at the family, make it more of a financial decision of investing money for your future wealth.

A couple of things to know about gifts…

One is “What is the lender’s definition of ‘gift?'” The lender’s definition of “gift” is using money that is not yours to buy your house. There are eligible donors of the gift. Typically, it needs to be a family relative of some sort.

Now, there’s also the IRS definition of “gift,” and that is a wealth transfer from one person to another. “It’s my money; I’m giving this other person my money,” is their tax implication. Obviously, I would always say to talk to your CPA and make sure that whatever is happening is to current code. But generally speaking, Mom can give you $5.4 million, and Dad can give you $4.5 million in your lifetime–$10.8 million the way the current structure is—without anyone paying any taxes, without the child paying any taxes. If they give you a gift that year, there’s a special tax form that they fill out that sort of goes against that lifetime gift exclusion.

It’s an interesting conversation about “What if Grandmom gives money? Does Grandmom have to give money to Mom or Dad and then Mom or Dad give money to the child?” Again, that’s what our handy CPAs are for.

You know, the coolest part about gifts and lending is that they’re becoming such the norm that things are getting easier and easier. If we’re already using current government guidelines to underwrite your income, where we soundly know you can qualify, really, everything is just about the equity position. “Is there enough equity in this property?”

Gifts for most investors, for most loans, all of the down payment can be from a gift—most, not all. If you get into the big, jumbo space, sometimes they’ll say, “If you put a gift, the buyer needs to put down at least 5% of their own funds.” Or some lenders would say, “Yes, you can use a gift for down payment, but you still need to meet our reserve requirements, which is money you have in the bank after closing, what-if money.” But generally speaking, it’s very easy to use a gift.

There are hard ways of documenting it and easy ways of documenting it. I think some lenders make it too hard. The hard way is telling my parents to give me a check, copying the check, then getting updated bank statements showing that that money is cleared, and then go to your mom and dad or whomever and get their bank statement to show that that money came from their account and that it cleared their account. It’s a real pain.

The easy way is wire transfer directly from Mom or Dad to the settlement company on closing day —really simple, and that’s it. Gift letter, wire verification—it’s super simple.

Now, one of the other areas where people pull down payment is just from raw investments.

You got a job, you opened up an E*Trade account, and you put $500 a month in some mutual fund, and that mutual fund grew over time. You really liked the iPhone, so you bought a bunch of Apple. You buy Amazon Prime for everything, so you own a bunch of Amazon stock. The idea is, “Should you liquidate those holdings and, therefore, use that money to buy a house?”

You have to look at each sale and say, “OK, if I sell stock, there might be a tax implication. Well, first, why am I selling it? Do I have any other options? Can I put less money down without selling? If the answer is no, and I need to sell it, what is my gain? What is the tax implication? Will I have enough money come tax time to satisfy whatever that tax implication is?” There’s a difference between long-term gains and short-term capital gains, and CPAs can help you with that. A loan officer should be able to help you with that, too, just by looking at your statement. But generally speaking, I’m more of a “Keep your money where it is, as long as it’s sound, and you’re comfortable with the investments, and try to borrow or smidge more out of your mortgage before you start liquidating other positions.”

Now let’s change gears a little bit.

Now you’ve figured out what to do, you bought your house, and you’re getting your mortgage. So, what do mortgage companies want to see? Obviously, we want to figure out where the money is coming from. “Well, first, do you have it, and where is it? Give me that bank statement or two. You either give me two consecutive or just give me your most recent. There can be differences there.”

And really, what they’re doing is looking at the beginning and ending balances and any deposit that seems crazy or weird—not payroll. The idea is sourcing or proving where that money came from. And it’s not all money. Some lenders go crazy overboard here, where you sold some concert tickets to some friends, you paid for all of it, and they each paid you $75, and you had four different $75 deposits. Some lenders will say, “I want verification of all $75.” That’s annoying. Hopefully, you’re using someone who doesn’t get that technical, and they have some specific rules to measure the deposits.

The other things that we want to make sure of while we have the money is that the money is yours or not. If it’s not, hopefully, it’s a gift. Or if it’s not, where did it come from? Just prove it. Then the other part is any recurring debts that appear to come out. This is where lenders typically find child support or IRS payments—anything that’s this direct debit that doesn’t necessarily show up on your credit report. They’re kind of looking at the money leaving your account. They’re certainly looking at the money coming in.

Again, just know that, if you’re out shopping around, you would just ask your lender, “Well, what exactly do you look for on my bank statement? What is your tolerance for what you consider to be a large deposit?” In many cases, it’s something that’s in excess of 25% to 50% of your monthly income.

One big thing that’s important to know when you’re getting a mortgage, and this has been a big change since the crisis… So, in the old days, you could have one really strong something overcome a weak something else. “I make a lot of money, but my credit stinks,” or, “I have a whole bunch of cash, but I make no money.” Someone could make a play and say, “Yeah, but…” they’re really strong with that thing.

In today’s world, everything is its own bucket. Credit is a bucket. Income is a bucket. Assets are a bucket. Valuation is a bucket. Everything has to be analyzed independently of the others. “I get you make a lot of money, but we still need to check off the asset box, and that box says, “We need to look at the statements and verify.” So, I would say, of all the things that we do with homebuyers, the assets is the one part that makes people most frustrated. For my high-net-worth clients, I usually say, “This is the part of the transaction that you’re going to start to hate me,” because if you move money around a lot, just be prepared to have all the statements. It’s really easy for lenders to just match it all up. If you have four different bank accounts, and you move money around to all four, just give them all four, but a lot of people try to hold back. Then when we say, “Oh, but I need to verify that thing,” that’s where the frustration comes from. Just know it’s sort of a regulatory “check the independent box,” and you might have a super high net worth or have the world’s best credit or make a whole ton of money, but we’re still going to look at the individual pieces of the statements.

Now, I know many out there would say (again, I hear this all the time), “I’m putting down $600,000—why is this a problem?” Sometimes it’s either, “Why are you asking for that other statement? Why are you asking for proof I liquidated that stock? Why do you care about that income document because I’m putting so much money down?” Again, just know that, because everyone is checking independent boxes, one strong thing can’t overcome another. Every little detail winds up mattering.

After listening to this podcast, there’s one thing that I really would like you to take away, and you’ll hear me say this over and over again: Every dollar you make or have can be earning or saving something. I think if you write that down and just look at every single transaction or decision you make through this buying process in determining everything, that sometimes brings to light what’s best.

If you’re thinking of getting a gift, “Well, hey, what are you doing with that money, Grandma? If it’s just sitting in cash, maybe you can give it to me, and I can buy a house.” “Hey, I’m thinking of selling that stock.” Well, if you sell that stock, it’s short-term capital gains, it’s ordinary income—that’s a 30% haircut right there. “Oh, I’m thinking about liquidating my Roth IRA. I don’t have to pay any taxes on that.” Yeah, but you’re going to lose all the future gain. What does that future gain look like? I think if you look at everything with that in mind, it should, hopefully, bring more clarity to what you should do.

So, through this podcast series, we’ve talked a lot about myths of home buying, what exactly is in the mortgage payment, and how much should you put down. Now, the next one is going to be very interesting—it’s “What should you buy?” It sounds easier than it really is, so be sure to listen to that. If you have any questions in the meantime, you can always email us at DownsGroup@VellumMortgage.com, or check out our website at www.DownsMortgageGroup.com.

Thanks for listening!

Special thanks to everyone who joined us. Until next time! Share your thoughts!

  • Leave a note in the comment section below.
  • Share this show on Twitter or Facebook.

In this episode, we’ll cover:

  • How much money do I need and where should it come from?
  • What are limiting factors and how do they affect my loan process?
  • What are some low down payment options?
  • Should I use some of my 401K money to put down on a house?
  • What do lenders want to see?

Got Questions?

I'm happy to answer your questions. Fill out the form for answers!

To help out the show:

  • Leave an honest review on iTunes. Your ratings and reviews really help and I read each one.
  • Subscribe on iTunes.

Listen to More Episodes

Mortgage Myths

Six Mortgage Myths

The Cost of a Mortgage

What's in Your Payment