The title pretty much describes it. So, should I connect my older, weaker modem (which in its own can broadcast a wifi signals) to a new router that can broadcast much better through an Ethernet cable? If so, would the signals get better or would it stay the same?
What you are looking for is an “Access Point”, this is basically like a router except specifically designed to connect via Ethernet and broadcast a WiFi signal, but not do any NAT or routing or modem-like behavior.
Another term to look for is “wired backhaul network extender”. Some ISPs may have a recommendation for specific devices to serve this role and known to interoperate with their “modem”.