When muslims speak of “people of the book,” to whom are they referring?

Not only Muslims speak of "people of the book"- this phrase is also known in Judaism (but in Judaism it only refers to the Jews). 

From the point of view of Muslims, this phrase refers all the Abrahamic religions:  it's Jews and Christians - the worshipers of God who don't acknowledge Muhammad. 

The three sociological perspectives include the beginner’s mind, culture shock, and the sociological imagination