In reply to Thugitty Jugitty:
Even if your house was perfectly sealed from the outdoors, it will still contain sinks and sources of water vapour. Therefore your activity and room temperature will influence the water content of the air. The fact that the dewpoint drops when you leave suggests that you and your activities increase the water content of the air. You might be generating water vapour by showering, washing up, drying laundry, breathing and sweating.
Because your house isn't perfectly sealed, there will also be exchange of air with the outside, so while you're not there the dewpoint inside your house should gradually approach the dewpoint of (drier) outside air.
Also dewpoint is not solely determined by air water content but also by pressure, so it would vary even in a perfectly sealed box of air with no other sinks/sources of water vapour in it.