Out-of-the-box, the Entity Reference module doesn’t provide a way to access the referrer entity’s fields from within the referenced entity. I have two ways of doing this. The first is to patch the entityreference module (not ideal unless this patch went out to the community via proper channels; perhaps I should get on that?). The second is to mimic entityreference’s “Rendered entity” field formatter but modify it to include the referrer information. I recommend the second method for now.