I take my EM-10 everywhere, multipitch, scottish winter, usually with the 9-18mm zoom but I may take a couple of extra lenses if cragging, like the 45mm 1.8
I plan to upgrade to the weather-sealed EM-5, but the EM-10 stubbornly refuses to break.
I certainly wouldn't describe myself as a photographer, but I do take a lot of photos when climbing, walking, kayaking, biking etc.
Firstly, I think its worth asking yourself a basic question - will you be going out to take photographs, or will you be going for a walk/climb etc and maybe take some photos while you're at it?
I am generally in the latter group, i.e. photography is not the main aim, generally.
However, I do occasionally go with photography as the main aim, and it is then that I take the dSLR and some decent glass (an old Olympus E3 in my case) tripod etc. As you suggest, a Nikon d3400 would be a good starter in this case, and has the added advantage of being able to shoot video too. Nikon/Canon also tend to have many more secondhand lenses available than the 4/3 lens mount family, if you want to expand your kit later.
On all activities I always take a compact 'tough' camera. I currently have a Nikon AW100, but may change soon to the Olympus TG5. The advantage of such a camera is that it can be hung around your neck and tucked inside jacket/fleece/t-shirt and whipped out for that quick shot of your heroic partner, when doing so with a dSLR might be less practical. A compact, easy to operate, weatherproof, shockproof camera will often enable you to capture images that you'd miss if reliant on bigger/better(?) equipment.
> For example the kit lens on a Nikon d3400 has comparable specs to the kit lens of the Olympus and yet is enormous in comparison...
The sensor in the olympus is smaller, thus the lenses needed are smaller. A 25mm lens on the EM10 gives the same field of view as a 50mm lens on a full frame camera. This is why many photographers needing to carry their kit around are downsizing from full frame or APS-C size sensor cameras to m43 cameras, especially if they carry many lenses. It's not a free lunch though - larger sensors exhibit less noise for the same exposure/ISO parameters and are better in low light for moving subjects. The image stabilisation in the OMD series can compensate for this for static subjects. Larger sensors can offer more megapixels, but that's only really an issue if printing massive prints or if you have to crop an image heavily. Look up 'sensor crop factor'
yes, all oly m43 users would love to have the 9-18 or 12-40 pro - they are good lenses- but expensive.
i'm using the oly 12-50 (w-sealed, bulky, nice macro), the panasonic 12-32 (sharper, tiny, but CA issues on older oly bodies) and the oly 40-150mm with my EM5. (all for less than either of above)