considering you have been having ongoing poop issues as well yeah i'd be very concerned about the management in your home when you are not there...is the sitter there all day while you are at work or shorter periods here and there? Though I will note that caring for 3 kids AND a young labrador puppy is really a big ask. Lab puppies can require as much direct supervsion as a young child but even more so with a toldder/young child moving around. (not sure quite how young your kids are - but looks like they are over 4 which helps!)
honestly i'd probably check with the vet. if the are acting normal, eating and drinking normaly and most importantly, pooping then i'd probably hold off (though you can generally chat with the clinic without rushing in too). I'd wonder about toxicity but if it was inhaled yesterday it's already been quite a few hours.
hope's he's good as new in a few hours