Category Archives: Musings

It’s winter, and the heater is not starting up!

Living near Chicago, it can get quite cold in the winter time. Just last week, the temperatures dropped below 0 degrees Fahrenheit. My furnace was working overtime keeping my house warm.

I’m a pretty warm blooded person, I usually run my winter thermostat around 64 degrees, a temperature that would be too cold for most people, but works well for me. As anyone who reads my blog knows, I have a lot of computer equipment, and as anyone who has owned computer equipment knows, it creates a lot of heat. Keeping my thermostat low helps keep my computer room comfortable and my server rack cool.

Despite this low setpoint, my furnace was struggling last week. Luckily, it made it to the finish line, but this week when temperatures rose to almost 40 degrees Fahrenheit, problems emerged.

I am a night shifter, and usually stay up until at least 4 or 5AM most nights. On one particular night earlier this week, as I was getting ready for bed, I emerged from my warm computer room and noticed the downstairs of my house felt chillier than usual. Sure enough, I checked the thermostat, and I was at 61 degrees, 3 degrees below setpoint. Uh oh.

I went down to the basement to check the furnace, it was running, but blowing cold air out the vents. I noticed a flashing indicator light blinking a 4 blink pattern, and after checking the inside panel, diagnosed that it was a “sensor over limit” error. I assumed this meant there was a bad sensor, probably reading 0 or infinity. I did not assume this would be a major repair.

However, by this time, it’s almost 6AM. I need to get to bed so I can get up for work, even for me, it’s now getting to be past my bedtime. But, although it’s 40 degrees outside, there are 20 degree lows in the forecast, I am not sure how my house will handle without a furnace as temps drop below freezing, and I know that when I get up and go to the office, I will not likely have a lot of time to talk to HVAC companies while they’re open. I think, it’s already almost the start of the business day, I can probably call someone now.

So I began to quickly try and find someone to call, I checked around online, but so many HVAC contractors feel shady or scammy to me from the get-go, I want someone trustworthy. My grandma recently had her system repaired, so although it’s early, I give her a quick call. She can’t recommend her guy because he doesn’t come out to my area, so the search continues.

After some deliberation, we decided to call the phone number on the last service sticker. Evidently the previous owner of the house had the system serviced in 2011 by Anderson Heating (ARS). Grandma thought that they were a good company, and they’d worked on the system in the past, so I decided they were my best bet. I gave them a call, and they were able to schedule an appointment for me that would work fairly well with my work schedule. I’d miss out on a few hours of sleep, but be able to address the issue that afternoon before going into the office.

The ARS Tech Appointment

The next morning, quite a bit earlier than the time we’d agreed upon, I received a call that ARS was on the way. I’d only slept about 2 hours at this point and I was not super thrilled, but at least I’d have this situation behind me soon, or so I thought.

When the truck pulled up, I immediately got some weird vibes. I noticed the name of the company was ARS Rescue Rooter, and the logo was different than the one on my service sticker from 2011. Had I called the right company? Was this one of those scams I had heard about where a company buys up a bunch of out of business contractor’s phone numbers to scoop up leads? (Fear of this type of scam was one reason I was so skeptical of calling someone off of Google.) My suspicion was high. I guess some of this is on me for not researching them ahead of time, but it was late and I was tired, and at the time I was doing what I thought was best.

The technician came inside, and he seemed like a nice guy, things seemed to be off to a reasonably good start. Though, I did notice, he immediately seemed negative about the outcome before even opening the panel, telling me that since the unit was 12 years old, we could be having problems.

Upon arriving at the furnace, he opened the panel up, and told me he would take a look at everything and should have some information in about 15 minutes, somewhat encouraging me to leave the area so he could work. I wish I hadn’t left him alone with the furnace now in hindsight, but that’s a totally normal thing to do when people are working on your appliances, and I still thought this was just an honest service call. So I went and got a drink, waiting around to hear some results.

When he called me back, he showed me some video from a small flexible camera. He pointed out some lines in a metal panel, stating that they were cracks in the heat exchanger. He proceeded to explain that replacing the heat exchanger is usually not worthwhile, because of the labor-intensiveness, and that his company is not legally allowed to turn the furnace back on with a defective heat exchanger.

A Cracked Heat Exchanger, Huh?

I now know that this is a common scam, so common in fact, that it appears on multiple websites about avoiding HVAC scams.

Here’s a quote from another contractor, Hot Point Heating and Air Conditioning LLC’s website, on the subject of avoiding HVAC scams:

Perhaps the most common furnace repair scam is the “cracked heat exchanger” scam. A shady repair technician may try to trick you into believing the heat exchanger in your furnace is cracked and leaking deadly amounts of carbon monoxide into your home. They will push you hard to either get it fixed immediately or replace the unit entirely on the spot.

Remember: it is possible for your heat exchanger to be cracked and it is a dangerous problem needing immediate attention. However, it is also very rare.

If your repairman claims to have found a crack in your heat exchanger, insist on seeing the crack for yourself—particularly if it took very little time for the repair technician to make the diagnosis.

If you cannot see the crack, do not take it on faith that only the expert’s eyes can see it. Get another opinion as soon as possible to make sure you aren’t getting scammed.

So, let’s do a litmus test here. Was I shown the crack?

Well, I was shown something. The technician had a live camera feed (I know for a 100% fact it was a real camera feed, not pre-recorded, because the technician accidentally dropped the camera at one point and I saw stuff going by as he did).

The camera feed did show some kind of black formation that appeared to be along a crease in the metal. Was it a dangerous crack in the heat exchanger, with the potential to release carbon monoxide? I’m not qualified to tell, this is why I have HVAC technicians working for me.

I suppose it might be a small hairline crack, I suppose it might also be soot buildup. Who knows?

But do I have a carbon monoxide leak? My carbon monoxide detector says: No!

ARS definitely provided some evidence of their diagnosis, and I am not saying that this evidence is fabricated. But, I think the diagnosis that the unit is dangerous is dubious at best.

A high pressure sales situation?

So now ARS had me backed into a corner. Obviously, I need to get my furnace running. The cold temperatures are coming back, next week is going to be even colder. They are essentially telling me that repair is not worthwhile.

The technician told me that my unit was unsafe to operate, and that I had to sign a form acknowledging this fact so that they would not be liable. He did not clearly communicate to me that I had the option to not disable the unit, and pointed to a section on the form and told me to sign it.

Later, when I reviewed the form closer, there was a second section I could have signed that said I accept responsibility and do NOT disable my unit. But, thanks to the pressure, and lack of sleep, I’d just authorized the gas to be shut off, the power to be shut off, and the unit to receive a big red sticker declaring it dangerous.

In the end, I feel that my choice to not disable the unit was not made clear, and that I was coerced into agreeing to disable it.

After signing this waiver, I am provided an enormous quote of around $2,500 (around what my grandma paid for her entire furnace when she had it installed) for all of the repairs that ARS says would be necessary to make my furnace safe again, including:

• Replace heat exchanger (~$1,500)
• Replace pressure switch (~$400)
• Replace run capacitor (~$250)
• Replace gas union (~$185)
• Parts acquisition fee (~$125)

Obviously, when these numbers start to add up, purchasing a furnace seems like a pretty good idea. Fortunately, my ARS technician is able to schedule a sales rep to come out that day and spec out my new system.

In the heat of the moment, with 2 hours of sleep under my belt, this felt like a good choice. I could get a quote, then think about it later at work. So I went ahead and scheduled the next visit. We schedule an appointment between 2-4PM, which is when my original tech visit was supposed to be scheduled.

I attempted to go back to sleep, and sent the repair quote to my parents along with a request to investigate if they feel like the company is legitimate while I’m trying to sleep before my second appointment.

The Sales Consultation

Later in the day, I was awoken at around 3:15PM by the phone. I’d set a last-minute alarm for 3:30PM, which would be just enough time for me to get to work, but expected to be awoken by the phone much earlier than that. I was surprised by how late it was.

The sales rep had gotten held up at another job and wanted to know if he could come at 5PM. Since I had to leave almost immediately for work, I told him that this wouldn’t work.

This turned out to be the best scheduling mishap that could ever happen, because had he come to my house that day, it’s quite likely I may have unnecessarily spent over $5,000 on a new furnace and A\C installation (of course, they pushed A\C too).

We agreed to reschedule to 10AM the next morning. This is too early for my liking, but we picked this time because it might allow for them to do an installation the same day if necessary and get my heat back up and running.

Reconsidering, A Second Opinion

In the time that’s passed, both myself and my parents who I’d enlisted to help research had begun to grow skeptical of the diagnosis from ARS. We decided that since I would already have to go a night without heat anyway, we may as well take our time and get another opinion. Afterall, we still had a few days of nearly 40 degree temps on the forecast, and if I could make it one night, surely I could make it two or three if I had to.

We called another local HVAC contractor who my dad had found, and scheduled for an appointment the next day at noon. (Remember, ARS was scheduled at 10AM).

I decided that it was best to not sacrifice sleep, and growing more and more skeptical whether I wanted to buy a furnace from these folks, so I called up the number I’d called originally to schedule my first appointment. I went through the phone prompts to update my appointment status, and spoke to a very kind dispatcher who informed me that as requested, my appointment had been cancelled. I told him thanks, and I’d create a new appointment soon if I decided to move forward.

An Unexpected Visit

After I cancelled my appointment, I assumed I might receive a call the next day to try to follow up. This would be a fairly standard practice I think for any business. But, I knew I needed my phone turned up so I could receive a heads-up call from my other contractor. So as a temporary measure, I blocked the office number of ARS, so their call would go to voicemail. I figured this would allow me to follow up later without my sleep being interrupted by a sales pitch.

Much to my shock, I received an awakening at 10AM in the form of a phone call from the sales rep. I answered the call and my Google Voice call screening confirmed his identity, but I let the call go to voicemail and went into the bathroom. I was considering what to do, I didn’t ask for this appointment to be completed, and I wasn’t sure I wanted to do business with this company after all.

I decided to call my mom quickly and decide how I should proceed. While I was on the phone with her, there was constant pounding on my front door and multiple calls spamming my phone.

I can forgive this excessive attempt at making noise — I do know the sales rep assumed that I was sleeping (I’d told him my schedule), and I assume he was trying to wake me up. But throughout the course of this, it seems like he realized that one of their numbers was going to voicemail, and proceeded to call me from two other numbers.

He’s now called me from both office numbers and his cell phone number, but doesn’t realize that the appointment is cancelled? How does the office not know this? You would think at some point during this situation, he would check the status of the appointment. Maybe something unexpectedly came up, maybe I wasn’t home, the whole situation just felt very off.

It became quite clear I needed to address the sales rep, but my mom told me I should not let him in the house, and she was beginning to get freaked out by all the noise while we were on the phone. I agreed and didn’t want to answer the door, this was starting to feel a bit too aggressive for my comfort. Especially when I looked at some voicemail transcripts, and I saw that he’d said “I can hear you in the house, why aren’t you answering the door?”.

Further, he was implying in the voicemails that he’d set aside his entire day for me and had no other appointments. It had a tone to it that seemed like he meant to make me feel bad. But did I feel bad? No. I cancelled the appointment, I provided notice, so why was this guy here waking me up pounding on my door? Is this not ultimately my decision whether to proceed with the sale?

Finally I got off the phone with my mom, and called the sales rep on his cell phone. I explained that I’d felt pressured into buying a furnace yesterday, and after having time to think, I wanted time to get a second opinion. He seemed OK with this, but reiterated how I was wasting his whole day and blowing him off. I explained that I’d cancelled the appointment, he claimed to know nothing about this, and he asked why I didn’t tell him directly.

Well, I’d never met this guy. He was scheduled to come out the day before, but how am I to know the company is sending the same rep? I spoke to him once on the phone and the conversation was about him being late. I didn’t realize he had personal attachment to this sale, companies send out different people for different jobs all the time, so I notified the company properly through their phone system and speaking to a representative. The fact this guy seemed to think I should feel obligated to tell him personally, and almost offended that I didn’t, felt off to me.

Was this just an attempt to manipulate me with guilt? Was the appointment cancellation genuinely not communicated to him by his company?

Either way, I decided this is not my problem, and this is not a company I want to be doing business with. At worst, maybe they attempted to manipulate me into a purchase I didn’t need. At best, maybe their internal communication between departments sucks. Regardless, why do I want to be involved?

The sales rep and I spent about 15 minutes talking on the phone, mostly he defended his company, asserting that they do things by the book and appealing to authority a little bit regarding how they are affiliated with Home Depot. We did leave things on good terms, at least I felt so, and he seemed to be genuinely surprised that I’d felt the way I did. I was quite up front with him about how I felt coerced and pressured.

A Real Repair

Later in the day, the alternative contractor we hired arrived. I did not tell them directly any details about the previous diagnosis, but they did know that another company had investigated and we were seeking a second opinion.

Before he arrived, I removed the “DO NOT OPERATE” sticker that I’d been coerced to have stuck on there. I wanted to let this tech make his own, unbiased, assessments.

He opened up the furnace, and I went through the same details with him that I did with the ARS rep, about what error code I was getting, the fact the furnace was blowing cold air, and questioning if a sensor could be bad.

Within 5 minutes, he had a jumper on the sensor, and the furnace ignited! We had heat blowing out the ducts.

After his investigation, he informed me that the error code was caused by a bad sensor, and that the sensor may have overheated due to restricted air flow. He believed that the filters I was using were too thick, and suggested using a filter with a lower MERS rating to prevent future sensor burn outs.

This company quoted me a total of under $300 to correct the problem. Of course I jumped on this, and the problem was fixed in under an hour.

As an added bonus, he told me that he could increase my blower motor speed, and that this might offset the thicker filter and prevent a future sensor burnout. I had absolutely no idea that this was something you could do! My air pressure on the second floor has always been lackluster, so I jumped on this offer. Now my air pressure throughout the house is remarkable. I look forward to seeing if this helps the A\C in the summer time.

After he was done repairing the problem, he seemed to do other tasks related to checking sensors and components of the furnace. He spent quite awhile checking things and I felt he was quite thorough. He spent a lot more time than ARS did before they declared the furnace a lost cause.

After he was finished, I asked him what his assessment was of the lifespan. As a 12 year old unit with an expected lifespan of 15-20 years, it’s not a young furnace. He said it looked like things were mostly in good shape, and he’d expect it to last closer to 20 years (so, I might get another 8 years out of it!).

I waited until after the transaction was over and I’d paid to ask him about the heat exchanger. ARS had made me so terrified to even say anything about it, because they told me that, in a nutshell, no legitimate company would repair a furnace with this defect and any company that would was risking serious liability.

He told me that if the heat exchanger were cracked, we should see flame rolling, or see carbon monoxide leaking. He observed neither of these things, and had measured the carbon monoxide himself during the diagnosis.

I told him all about my experience with ARS, and if anything, he spent most of the time rolling his eyes. I told him what they said about liability, and that I’d never find anyone legitimate to fix it. He seemed almost in shock at this.

The Takeaway

For me personally, ARS is not the type of company I want to do further business with, and I will not be calling them again. I will be calling the contractor who did fix my problem for my future needs.

So what’s the deal, is ARS scamming people with heat exchanger ghost stories? There seem to be a lot of reviews by other ARS customers mentioning bad heat exchangers. But, these customers seem mostly satisfied, having the tone of “maybe ARS caught a serious problem I didn’t know about”.

I won’t make a hard claim one way or another. Perhaps there might be a hairline crack in my heat exchanger. But if it is not leaking, is it a serious problem? My furnace is up and running, and a competent HVAC technician gave it a clean bill of health.

I can understand that ARS may be technically correct, while also being morally in the wrong. I feel that they exaggerated a minor, age and wear related, symptom that is common in furnaces and blew it up into a major life or death situation in order to use the cold winter temperatures to force a quick sale.

I also feel that they must do this a lot… not just because of the reviews we read, but also because of how readily the tech whipped out these forms and went through his script.

I think about how someone with less knowledge and experience might react to these tactics. The previous owner of my house was an elderly lady, and I wonder if the furnace that is now in my house was installed as a result of this kind of sales tactic. I hope she was treated fairly by them in 2009.

Could this have been one bad apple, one bad technician? It seems unlikely, because my sales rep, while defending the technician, expressed to me that he is one of their senior technicians, and that he trained the technician personally. So, if their senior technicians are operating this way, it’s likely they are training their subordinates to do so as well.

Conclusion

ARS got $95 of my money for a service call where they did nothing but show up, did not disclose to me what the actual problem was (at no point during that visit did the tech mention the bad sensor), and I feel used scare tactics to try to sell me a new furnace which I clearly do not need.

Still, I saved big by not falling into the trap.

I decided to use my extra money I saved by not buying a furnace to buy a new carbon monoxide detector which I will install in the basement, just in case. My existing detectors were over 5 years old.

I feel much better about this solution than trying to do an emergency replacement of the entire system in the middle of winter.

If you believe the heat exchanger fear hype, perhaps I’m now at a higher risk of a carbon monoxide leak? But, this has caused me to ramp up my protection by buying a new detector, so I feel that I am now MORE safe and LESS likely to be poisoned by carbon monoxide. Any appliance that burns fuel can spring a leak, “hairline crack” or not. Monitoring is the key to not succumbing to carbon monoxide poisoning.

I feel that the other contractor I hired was competent, fixed the problem correctly, and did a thorough inspection of the furnace. My furnace now operates better than before it broke, thanks to the increased air pressure, and the problem is fixed for much less money than ARS quoted me, if I had elected to repair with them.

Ultimately, I think everyone’s takeaway here should be, if you get an astronomical quote from any appliance repair company or if you feel that you are being pressured into a sale, second guess them! Get another opinion! You may come out very far ahead.

If you are following my articles here, you may have heard that I recently got Comcast Gigabit Pro and now have 2Gbps Internet at home.

Many have asked me how I am accomplishing this internally and what approach I have taken to getting multi-gigabit networking up and running in my house, so without further ado…

My Edge Network Cabinet

This cabinet is where my Internet connection enters the house from Comcast. The top two rack U’s are Comcast-owned equipment.

• Fiber Patch Panel – Where the Comcast fiber terminates inside my house.
• Juniper ACX2100 – Comcast’s fiber CPE, acting as just a switch to pass the handoff to me.

My equipment in here consists of:

• A PDU for nice fancy cable management. (UPS that powers this rack is located elsewhere).
• Ubiquiti EdgeRouter Infinity (ER-8-XG).

The EdgeRouter Infinity has 4 ports in use. I ran two 10G fiber connections to my core networking cabinet, for reasons I will explain later. There is one 10G fiber connection to the Juniper and one 1G ethernet connection to the Juniper.

This is because the Comcast Gigabit Pro service provides two circuits, a 2Gbps fiber and a 1Gbps ethernet with different IPs. The EdgeRouter Infinity is my termination point for both of these connections and it holds both public IPs.

I use Policy Based Routing to direct traffic to the desired connection. Currently, I route only my DNS resolvers, VoIP, and Twitch RTMP (broadcasting) out my single gigabit ethernet connection. I also have an off-site video recording solution for my security cameras which I have routed out this connection. This way those critical, and fairly “consistent” usage, services will not be degraded by surges of traffic on the fiber line. This is completely unnecessary but it seemed like a good use of the extra capacity to me. At any time I can add an IP to my NAT group that I set up and route it out the connection of my choice.

My Core Network Cabinet

My core network cabinet is the next step, and main distribution panel for Internet and network connectivity in my house.

It contains the following equipment:

• 2 Obihai Obi200 VoIP adapters for my two Google Voice lines.
• Raspberry Pi for Bonjour relay between VLANs. (Helps the Chromecasts work as expected across my multiple VLANs.)
• 3 CAT6 patch panels for ethernet distribution. The cables on the other sides of these panels go to wall jacks and installed mounted equipment throughout my house.
• My old EdgeRouter ERPro-8 that I haven’t deracked yet, I used to use this as my core router for my cable Internet lines.
• 48 Port Ubiquiti EdgeSwitch (ES-48-Lite), for 1Gbps ethernet distribution to general purpose wall jacks.
• 24 Port Ubiquiti EdgeSwitch (ES24-500W), for 1Gbps POE ethernet distribution to IP cameras and wireless access points.
• Several small project computers (similar to NUCs) for seasonal projects.
• Ubiquiti EdgeSwitch 16-XG (ES-16-XG), for 10Gbps network distribution.
• PDU for fancy cable management.
• UPS that powers both this and the Edge cabinet next to it.

The bread and butter in this cabinet is the ES-16-XG which acts as a core 10Gbps distribution switch for the rest of my network.

While I do have two separate physical networks for server rack and residential network, separated by VLANs, there is a 2x10Gbps fiber connection going to my server rack from the ES-16-XG, which provides inter-vlan connectivity between my servers and the rest of my network.

Additionally, there is a 2x10Gbps uplink to the 48 port switch. The POE switch does not support 10G, so it has a 2x1Gbps uplink to the 48 port switch. This means the theoretical maximum throughput on my WiFi would be 2Gbps, but that is also the maximum speed of the fiber drop that I am using as my primary Internet connection, so this is acceptable to me.

10Gbps to My Computers

The remaining 10G ports on the ES-16-XG can be used for 10Gbps network connectivity of other devices. I have a total of 3 computers on the second floor of my house that are connected with 10Gbps currently.

• My main Linux workstation.
• My gaming PC.
• My DJ studio computer where I broadcast to Twitch.

This can be accomplished by using the 4 RJ45 ports that come on the ES-16-XG, but I didn’t want to be limited to 4 ports, so I also am using SFP-10G-T transceivers from FiberStore, they are currently selling for $59 new.

I haven’t really encountered any significant difference between this and the onboard 10G ethernet ports. I plan to continue to use these to expand the 10G connectivity as needed.

I have no problems running 10G over my existing CAT6 wiring in my house that I ran 4 years ago. I am able to get about 9.6Gbps on iperf from the second floor computers to the server rack in the basement.

To accomplish this on the computers upstairs, I am using Intel X540-T2 network cards I bought on eBay, they are currently going for about $130 used.

So, it is somewhat costly to get 10G to a computer upstairs via ethernet, but within my price range and working just as reliably as standard 1Gbps ethernet for me so far.

10Gbps to My Servers

For my server rack, I am using a Cisco Nexus 3064-X (N3K-C3064PQ-10GX) as the core switch.

This was obtained on eBay for under $400, and actually was cheaper than the EdgeSwitch (because I bought it used), but not ideal for a core residential switch because it is to deep for my wall mount network cabinet. For the server rack though, it works great and provides 48 10G ports (and 4x 40G ports).

As mentioned earlier, it has 2x10G fiber uplinks to my residential network, and also has its own 10G uplink to the EdgeRouter. These are all fiber links.

For the servers themselves, I use direct attach copper cables like this. This seems to be the cheapest way to do 10G over short distance and was ideal for my single server cabinet.

In a number of my servers, I am using Dell Mellanox CX332A cards, but I did run into some motherboard compatibility issues and also have two servers that are using Intel X520-DA2 cards, because for some reason even after BIOS updates and other troubleshooting I could not get the Mellanox cards working in all of my systems.

I do not notice any difference between the different 10G cards or media… DAC, ethernet, fiber – it has not made any difference that I have noticed and all performs well.

My server rack contains several storage servers which I have connected two DAC cables (2x10G) and bonded them.

For my other servers, hypervisors\compute, I have 2x10G as well but I have it set up as “public” and “private”, where the public port has a routable IP that can lead to the Internet, and the private port has a non-routable IP on a separate VLAN that is used for inter-server communications, mostly accessing NFS mounts on the storage servers.

In practice 10G is well beyond what I need, so this is still over-built for my purposes. Prior to upgrading my Internet, I was doing the same with 1G ethernet, and actually did not even have the storage servers bonded.

As you might imagine, my growing rack of server equipment consumes some serious power, so to that end, while I was having some electrical work recently I had a dedicated 120V 20A circuit installed for the rack. I also had a second one installed for future expansion.

What Am I Doing With All This Capacity?

There is no doubt that I still have much more bandwidth and capacity than I could ever use — and I am a heavy power user on a good day.

Having this kind of network and connectivity to the Internet has allowed me to free myself from any worry of excessive bandwidth usage. I no longer need ask if I can attempt a project due to bandwidth limitations.

The greatest limitation to me now tends to be external services. VPNs, servers, and connections outside of my control. Maxing out my Internet is basically impossible, even when downloading games from Steam (which is probably the application that comes the closest to doing it).

The best thing for me to come out of this connection is the freedom to use it however I want without problems. I need not worry about backup scripts running while I’m gaming, or Steam downloads choking out my Twitch streams.

My peak usage amounts that actually show up on my graphs are typically still under 1Gbps, and in many cases under 100Mbps, but the ability to do anything anytime without fear of choking due to node congestion or RF interference of cable service is very freeing.

The service is expensive, but for me it’s money well spent.