Nov 3, 2023

Using Sitecore Search REST api in NextJS

| Reading Time: 5 Minutes, 16 Seconds

2023-11-03

Our sites are often compositions of data from multiple sources. A common scenario is that product data are actually managed in a PIM system, maybe enriched in CMS. Back in the days we often imported products into Sitecore CMS, but with our composable architecture we want to limit that. One way could be that the product component requests the underlying system(s), but we want to link and show teasers etc. in other parts of the site. Sitecore Search can actually also help with that. The product pages can be indexed with the final result (or even pushed to Sitecore Search).

There is a nice REST api for Sitecore Search¹, when using the Sitecore Search SDK React components the REST api is called client side by the components, and hereby it is simple to see the actual requests to the api.

In Sitecore Customer Engagement Console where you manage Sitecore Search, there is in Developer Tools also a nice API Explorer that allows you to build search requests skeletons, write actual requests and test them live, so you can easily test your search requests.

Screendump of API Explorer from Customer Engagement Console

Authenticating REST API

To make requests to Search REST API you need to authenticate, this is nicely following the de-facto standard with Bearer tokens, and a when you get a Bearer token there is also a refresh token.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


  async fetchNewToken() {
    const now = Date.now() / 1000;
    const url = `${this.host}/account/1/access-token`;
    const response = await fetch(url, {
      method: 'POST',
      headers: {
        'x-api-key': this.apiKey,
        'Content-type': 'application/json',
      },
      body: JSON.stringify({ scope: ['discover'] }),
    });
    this.token = (await response.json()) as TokenResponse;
    this.token.accessTokenRefreshedAt = now;

    return this.token.accessToken;
  }

When testing a new API I always start fiddling with the endpoints, once I used Postman, now I always use the RestClient for VS Code where you can easily create a .http file and have your tests in repository.

Screenshot from VS Code with request to fetch token for Sitecore Search

The result is a standard Json Web Token (JWT) and we notice the expiry is included in the result. It is following the same notation as the token itself where the exp attribute is “number of seconds since Unix epoc” and here the expiry result attribute specifies the number of seconds the token is valid for.

We can easily see the decoded content of the token with jwt.io

As we get the expiry easily available in the result there is actually no need to read the actual content of the JWT we can just save when the token was fetched and compare the number of seconds.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


  async getValidToken() {
    if (!!this.token && this.currentTokenIsValid()) {
      return this.token.accessToken;
    }

    return (await this.fetchNewToken());
  }

  currentTokenIsValid() {
    return (
      !!this.token && !this.isExpired(this.token.accessTokenRefreshedAt, this.token.accessTokenExpiry)
    );
  }
  isExpired(requestedAt: number, expiry: number) {
    const now = Date.now() / 1000;
    const expiresAt = requestedAt + expiry;
    return now > expiresAt;
  }

The expiry is actually valid for one year, so in reality you can probably keep the bearer token. There is a high chance that your application is restarted or rebuilt within a year, (especially when using Vercel to build upon publish), but as always it might change in the future how long the tokens actually are valid for.

Making requests

With a valid authentication token, next step is to request some real content from Sitecore Search.

API Explorer in Sitecore Engagement Console used to create a request to Sitecore Search

The API explorer only supports adding search phrases. The url of the current page is necessary for the context for the context aware intelligence in Sitecore Search and to determine language etc.

Based on the documentation we can also specify locale as context instead and filters can be specified.

Request in API Explorer with filters and locale context

The widget reference is still important, as this is the link to the Sitecore Search widget where rules can be configured. It actually makes sense to create specific widgets for various usage so you can have distinct analytics and can fine-tune the rules and results in Sitecore Search.

Hereby we can make the actual request for Sitecore Search

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48


  async getFilteredItems(
    filter: object | null,
    pageSize: number,
    locale: string,
    widgetId: string
  ): Promise<T[]> {
    const token = await this.tokens.getValidToken();
    const body = {
      context: {
        locale: createLocaleContext(locale),
      },
      widget: {
        items: [
          {
            entity: this.contentTag,
            rfk_id: widgetId,
            search: {
              content: {},
              limit: pageSize,
              filter: filter,
            },
            sources: searchSources,
          },
        ],
      },
    };
    const response = await fetch(this.discoverEndpoint, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${token}`,
        'Content-type': 'application/json; charset=utf-8',
      },
      body: JSON.stringify(body),
    });

    const result = (await response.json()) as SearchResponse<T>;

    if (!result?.widgets?.length) {
      console.warn('Unexpected result from search', result);
      return [];
    }
    const widget = result.widgets[0];
    if (widget.errors) {
      throw widget.errors.map((x) => x.message).join(' and ');
    }

    return widget.content ?? [];
  }

Create NextJS component to show Sitecore Search

Now we have the data from Sitecore Search, we can create a Sitecore component to render those data.

With the Sitecore GetStaticComponentProps method we can handle data fetching even in a component and still support Static Server Generation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


export const getStaticProps: GetStaticComponentProps = async (
  rendering,
  layoutData
): Promise<ServerProps | null> => {
  const language = layoutData.sitecore.context.language;
  const datasource = rendering.fields as unknown as ProductListDatasourceTemplate;

  const title = datasource.Title?.value as string;
  const ids = datasource?.Products?.map((x) => x.fields?.ProductId?.value as string) ?? [];
  const filter = searchClient.createAttributeFilter('product_id', ids);

  const items = await searchClient.getFilteredItems(
    filter,
    datasource.Products.length,
    language ?? 'en-us',
    datasource.SitecoreSearchWidgetId?.value?.toString() ?? 'rfkid_products'
  );
  return {
    items,
    title,
  };
};

Those server side fetched data are available in the actual component with the useComponentProps method and hereby it is just a matter of handling the rendering

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30



const Item = ({ item }: ItemProps): JSX.Element => {
  return (
    <li>
      <a href={item.url}>
        <img src={item.image_url} alt={item.name} />
      </a>
      <span>{item.name}</span>
    </li>
  );
};

export const Default = (props: ComponentProps): JSX.Element => {
  const data = useComponentProps<ServerProps>(props.rendering.uid);
  if (!data || data.errors) {
    return <div>Product list error</div>;
  }

  return (
    <div>
      <h2>{data.title}</h2>
      <ul>
        {data.items &&
          data.items.map((i) => {
            return <Item key={i.id} item={i} {...props} />;
          })}
      </ul>
    </div>
  );
};

We can have even more control in Sitecore Search

Now this was actually a very traditional approach where an editor chooses which items to show, and an editor is required to update the content. You can actually move this responsibility to Sitecore Search, eg. specify a certain widget and have rules configured for the widget instead. It might depend on your actual organization.

Sitecore Search REST API documentation ↩︎