For federal income tax, married filing jointly is almost always a better choice than filing as two singles.
Additionally, there are other entitlements that marriage confers: right to attend and make decisions in health emergencies, shared life insurance and health insurance at lower total premiums (and depending on situation tax benefits as well like for a joint Health Savings Account).
Again, these are arbitrary benefits conferred by law. None of these currently apply in my jurisdiction*, but there's nothing whatsoever to prevent my government (or yours) from making marriage entirely benefit neutral, and certainly nothing to prevent them from making it entirely tax neutral.
*All costs and benefits of marriage here are, in law if not always in fact, automatically applied to 'de-facto' relationships - our government declares as 'de-facto' all people who "have a relationship and live together as a couple for two years or more but are not married". It's even possible to register a de-facto relationship with various agencies to ease the process of claiming those benefits, such as attendance and decision making in health emergencies, without getting married.