If you mean on all nameservers authoritative for a given zone, then yes they need to share the keys, since they are published as DNSKEY records, which will be used by recursive validating clients to check the signatures.
Now, nameserver A could publish DNSKEY of ZSK A + all RRSIG done with ZSK A, and nameserver B could have ZSK B + all RRSIG done with ZSK B.
And each one with DNSKEY of the KSK and the RRSIG with the KSK of the relevant ZSK.
For DNSSEC to validate a nameserver needs to find at least one correct path of keys and signatures. So, theoretically, a recursive nameserver hitting only nameserverA and retrieving all records from there, could get from there everything it needs to validate records. Same if it hits only nameserverB.
The problem however is with the only part and the fact that things are cached and that recursive nameservers do not "lock" themselves to one authoritative nameserver, but will switch depending on various algorithms. The resolving nameserver may have in its cache a mix of data with, for examples, DNSKEY A and RRSIG from B. Then, the validation will fail.
So, in short, I would be very much surprised if this works reliably.
This is discussed in length in this document: https://datatracker.ietf.org/doc/draft-huque-dnsop-multi-provider-dnssec/?include_text=1
It tries to see how a domain name could use more than one DNS provider at the same time for resolution, but still have proper DNSSEC management, with various scenarios depending on who does ths signing.
You could test that, but as Håkan said it seems you have another problem before coming to this problem.