I'd definitely talk to the city/county to determine if the lot is buildable. Dustin's correct, it could very easily not be legal to build there. Or once the setbacks are satisfied the resulting house may only be 4 feet wide or something silly.
Secondly, I'd check with the city/county/state to determine how you can build on a marsh legally. Some places won't let you due to wetland ordinances. If nearby houses are on stilts then it's likely doable.
Thirdly, if you intend to rent it out later, verify that the municipality will allow rentals in that area.
If that all checks out, I'd be pretty tempted to build on stilts as well. It's probably the lowest impact way to put a structure in place on a marsh. Luckily a tiny house won't weigh that much and the piers/stilts may not be big or expensive.
Actually, one other thing to think about is how stilts are installed there. The building inspector could be able to tell you. If a big outfit needs to get in there to drive the piers 30' deep, that might be hard or expensive to do on a narrow lot.
Maybe, once you start building your tiny house, the "large conventional" neighbors will wish they had bought the sliver and may offer you a higher price than what you paid to keep your "shack" away from their nice starter castles. Yay!
Sell it for a $20K
profit and get another lot somewhere else.