Instrument tracks in Studio One are like any ordinary pure MIDI sequencer track (before audio workstations). The instrument track can send MIDI to any instrument in the instrument rack and the audio channels in the mixer come from the instruments in the rack, not the MIDI track in arrange. They are not directly connected.
Muting an instrument track only mutes the MIDI playing from that track, not the audio channel in the mixer. Likewise, muting an instrument audio channel in the mixer only mutes the audio in that mixer channel, not any MIDI track assigned to it.
This is not a bug. This is how the Instrument Rack design works.
The audio channels in the mixer above come from the instruments in the rack, not from the arrange tracks, so muting of either thing is independent.
Below you see MIDI synths in a rack being fed by a sequencer. If you mute the sequencer tracks feeding those synths, it obviously doesn't mute the audio channel in the mixer or vice versa as they're two completely independent things.
Some users may prefer the other inline design where things always have a 1-to-1 relationship, and that's fine. There are subjective advantages and disadvantages to both designs.