Upon opening the Item Shop, the main Item Shop window will appear. On the left hand side (A) you can choose the different categories of items and if you then click on one of those categories, the items that currently are available in that specific category will appear in the main frame of the Item Shop window.

The yellow numbers at the bottom of the middle part of the window show how many pages of items this category has - you can jump to other pages by clicking the numbers (B).