My husband is non-exempt. He works for a company that does not pay overtime if a holiday or a vacation day was taken during the week.
For example, he may work 36 hours in 3 days, take an 8 hour vacation day, work another 10 hours on the 5th day. His total hours that week is 54. All of these days are on the same pay period. His company pays straight time for all 54 hours.
Is this legal?

Not quite. Overtime is only legally due if the employee worked over 40 hours in the workweek. Paid time off hours need not be including in determining whether the 40-hour threshhold has been reached. In the example you described, there would be 40 hours regular, 8 hours vacation (at straight-time) and 6 hours overtime at time-and-a-half, assuming all this time was in the same workweek (see below).

One thing you haven't told us though is what the company defines as the workweek, which must begin on a date/time certain each work. The "pay period" is irrelevant.

I'm also assuming that the company is even subject to the Fair Labor Standards Act. Does the company do at least $500K in annual gross revenue OR is it engaged in interstate commerce OR is it a government entity, hospital/nursing home, school/college/university?

The company's work week is from Thursday to Wednesday. My example goes from Monday to Friday.
This is a large national company.

Then you have to count from Thursday to Wednesday, not Monday through Friday.