Google Search Appliance Solution Design and Planning March 2014
© 2014 Google
1
Solution Design and Planning This paper covers solution design and planning of a Google Search Appliance (GSA) implementation.
About this document The recommendations and information in this document were gathered through our work with a variety of clients and environments in the field. We thank our customers and partners for sharing their experiences and insights. What’s covered
This paper covers core planning activities with fundamental guiding principles that apply to all search deployments.
Primary audience
Project managers and GSA administrators
IT environment
GSA and various content sources and security mechanisms
Deployment phases
Deployment planning
Other resources
● ● ● ● ● ●
GSA Notes from the Field: Deployment Architectures GSA Notes from the Field: Security Learngsa.com provides educational resources for the GSA. GSA product documentation provides complete information about the GSA. Google for Work Support Portal provides access to Google support. Google for Work Connect
2
Contents About this document Chapter 1 Overview Chapter 2 Capturing Requirements User requirements Content and security requirements Resource allocation requirements Performance and scalability requirements Failover / High-Availability Administration and reporting requirements Chapter 3 Identifying Phases Deployment phases Where to start Phasing your activities How long should phases be? Chapter 4 Defining Success Criteria
3
Chapter 1 Overview A successful search solution is conceptually very simple: help users find the information they are looking for. Make search fast, make it easy, and make it relevant. The Google Search Appliance takes care of the speed, ease and relevance. But you need to plan and execute the project to take full advantage of the power of the search appliance. Key to this approach is remaining focused on short delivery cycles and structuring work around these phases. Every deployment of a Google search solution is unique. In one use case, you might be providing search across SharePoint content and extending core search with purchase orders from SAP. In another, you might be providing search of the millions of documents that businesses can accumulate over time, bringing them together with policy documents, and the contact details of the people who wrote them. Although each deployment has different content sources, security requirements, and user needs, there are core planning activities with fundamental guiding principles that apply to all search deployments. This paper focuses on the following core planning activities:
● ● ●
Capturing requirements Identifying phases Defining success criteria
4
Chapter 2 Capturing Requirements As you capture requirements, group them into related sets that you can prioritize and align with phases of work. In general, focus on the following areas:
● ● ● ●
User requirements Content and security requirements Performance and scalability requirements Administration and reporting requirements
User requirements Understand what is important to make the deployment successful from the user perspective. In general, user requirements focus on:
● ● ●
Usability Breadth and depth Communication and feedback
Usability For users, search should not be a chore. Defining usability requirements can help ensure that users find your search solution intuitive and effective. As you identify usability requirements, consider the following issues:
●
What are the usability features that really make the search solution resonate with users? ○ The Google Search Appliance offers many simple-to-implement, on-box usability features, such as query suggestions, KeyMatches, Dynamic Navigation, Entity Recognition and Expert Search to name only a few.
●
How do users access search? ○ Choose the point of access of the search so that it is most convenient for users. The search page could be accessed directly on the GSA or indirectly through a webapplication that is parsing the results from the GSA XML. For indexed portals, it might make sense to replace the existing search-interface with Google search (for example, Search Box for SharePoint). ○ Note that there might be several entry points to access the search. One might be the Intranet, another one in a specific Web application, a mobile app or even a thick client. Different access points might also increase complexity as they could require additional authentication mechanisms.
●
What speed requirements do users have? ○ AJAX style technologies can dramatically enhance perceived responsiveness and performance, while providing a richer search experience.
5
In general, meet usability requirements as early in the release cycle as possible because these are not typically tied to content sources and they can get users excited about the search solution.
Breadth and depth A search solution needs to meet the demands of your user community. Defining breadth and depth requirements help ensure that you have covered a wide enough user group while providing the right content for them. As you identify breadth and depth requirements, consider the following issues:
●
What do the user groups look like? ○ Where possible, the largest groups and the users who are experiencing the most frustration today should be brought on first. ○ Using search appliance front ends, you can present a different look and feel and different content to various users, based on their needs. For information about front ends, see Creating the Search Experience.
●
What are they trying to find now, but are frustrated that they can’t? ○ This is the content that should be in early phases. ○ User onboarding should be aligned with the inclusion of the content they are looking for. That is, try not to give the users a search capability before including the content they will be looking for.
●
Do some users have more sophisticated search needs? ○ Out-of-the-box advanced search can easily be augmented through the rich use of metadata and other core functionality. ○ Where this is not required, keep search simple, but functionally rich.
Communication and feedback User feedback is an effective tool for identifying usability issues. When you solicit feedback, you let users know that their opinions about the search deployment are important. Defining communication and feedback requirements ensures that you give users the ability to provide input to the implementation team on the search deployment—what is working for them and what is not working. As you identify communication and feedback requirements, consider the following issues:
●
What do you need to communicate to users? ○ In addition to adding new content and exciting new features, it’s important to make sure you tell your users about these to keep them excited about the product, and get kudos on your successes.
○
Because most of your users already know how to use Google search technology, training needs are typically minimal, but make sure your users know that they can now search enterprise content with the same ease as they search the internet at home.
6
●
How will you get feedback from users? ○ Having a search admin regularly checking GSA reports to gather statistics on: top queries, queries that return no results, and so on, to gain input into tweaking user experience is just one way of getting feedback on search. (Could be automated via Admin API)
○
User feedback is one of the best measures of success. Consider conducting periodic surveys with user groups.
○
Consider providing a feedback link for users (e.g., integrating with your existing ticket-system).
○
Place a “Did you find what you were looking for” form on the search results page so users can directly submit feedback.
○
To help capture user feedback, you can also use the free Search Quality Feedback toolkit
Content and security requirements For most organizations, the following two aspects of a search deployment typically go hand-in-hand:
● ●
Content Security
Scenarios that encompass content and security can range in complexity from completely unsecured public website pages to complex integration with an Enterprise Resource Planning (ERP) system such as SAP or PeopleSoft, and everything in between. Plan your end-state architecture in the early phases, but also phase in both content and security. In other words, don’t delay delivering a great search experience to your users because you want to index every last piece of content or implement a security framework that the user won’t need until later.
Content In general, analyze all potential repositories of organizational information as early as possible in a project. Although the Google Search Appliance excels at providing powerful, fast, and relevant search across unstructured content, you should not exclude structured content, such as your data warehouse, transactional systems, and so on. It is important to understand how content sources relate to each other, as this will help you define how to phase deployment of content. For example, content from a case management system may be supplemented effectively with content from a product catalog, enabling users to see not only product information, but also the types of problems and issues that users encounter when using the products.
7
The following table lists various types of structured and unstructured content sources and considerations that can help you define how to phase its deployment.
Content source
Structured/
Unstructured
Complexity
(Low/Medium/Hig h)
Consideration
Public websites
U
L
●
HTTP
File systems
U
M–H
●
SMB (Connector) or HTTP
Databases
S
L–H
● ●
Use database connector or a feed, or web enable Volume of content
Intranet websites
U
L
● ●
Might need to consider security Handling of dynamic pages
Staff portal
U
M–H
●
Might need to consider security complexities Might need to account for nonunique URLs (the same URL containing different content, based on user role)
●
CMS
U
L–M
● ● ●
LOB applications (for example, Lotus Notes)
S
Enterprise applications (for example, ERPs)
S
Other transactional systems
S
L–H
● ● ●
M–H
● ● ●
L–H
● ●
Might have metadata to leverage Need to determine if it can be crawled natively Might need to consider security If web enabled, might be able to crawl these content sources Might require a connector Security Need to identify core data Might require feeds or a connector Security Might be accessed by a custom connector or OneBox module Security
8
Security Security can be the area of greatest complexity in a search deployment. As you analyze content, understand if it is secured, and if so, how it is secured (forms-protected, cookies, protected by application-level security, and so on). Especially for projects that involve several content sources, it is important to have a deep analysis of the security options before the implementation starts. This is important as certain information about security can change the scope of a project drastically. So make sure you capture the security analysis and requirements up front. In a search solution, security has two main areas of impact:
● ●
Crawling and content acquisition Serving results—User authentication and authorization
For comprehensive information about the search appliance and security, see Managing Search for Controlled-Access Content. Crawling and content acquisition The Google Search Appliance can make use of standard security protocols, such as LDAP, HTTP Basic, NTLM, Kerberos, certificates, SAML or forms-based security. Understanding all the security permutations will help you plan for content acquisition. For example, security might have an impact on web and file system crawl that you need to plan for, such as configuring a proxy or ensuring your Windows file systems have CIFS enabled to support SMB indexing. More complex security might require alternative means of content acquisition, such as feeds or connectors. In most cases, the GSA will be well prepared to index standard setups. However, in some cases, you might find that you need to make small adjustments to your environment to enable the search appliance to crawl and acquire content without needing to use feeds or connectors. For example, you might find that to extend the crawl to a new subdomain, you need to modify a cookie domain as the search appliance crawls content to allow cookies to conform to request for comments (RFC) specifications. These types of changes are typically small and can be implemented through a variety of methods. Occasionally, security considerations require making adjustments to the indexing procedure or using an alternate content acquisition approach. These circumstances might affect aspects of the solution design that are not directly related to security architecture, such as feeds and publishing workflows. Content acquisition should be analyzed together with user authorization, as methods of authorization can differ between early and late binding. Generally, early binding (using ACLs) can be considered a quicker way to return search results than late binding, because with early binding the security permissions are stored on the GSA. Early binding therefore needs to cache the permissions on the GSA, which means that the information must be made available for the GSA. This ACL information can exist in feeds, in the HTML-header information or via the servers’ HTTP headers. Late binding is
9
the check when search results are going to be verified against the (usually) source system to ensure that the user has access right now, so no customization on the content is required. For more information, see GSA Notes from the Field: Security. Serving results—User authentication and authorization When serving secured content, the Google Search Appliance first checks that the user is entitled to see relevant results. If the user is not entitled to view a document, it does not appear in the result set. Keep in mind that returning secured results is usually handled by two processes, authentication and authorization: ●
Authentication verifies if the user is really who he pretends to be. During this process the GSA also assembles all relevant group information for this user.
●
Authorization uses the user ID to make sure that the user really has access to the particular content piece.
Understanding this concept is critical when several secured content sources have to be indexed. For example, only one authentication mechanism might be required for two secured content sources that both use the same user ID. However, if both content sources require different authorization methods, the GSA will have to be configured to use two authorization methods (for example, ACLs for one content source and connector authorization for the other). Of course, you can always choose to make results public and apply no security at serve time. In many cases, search can initially be deployed unsecured, with security added as more content is acquired. Public search (such as an externally facing internet site) is typically deployed this way. Out-of-the-box, the Google Search Appliance integrates with many different security mechanisms and can cater to various heterogeneous security requirements, including silent authentication (for example, via cookies or Kerberos) and integration with a single sign-on (SSO) system. For customized integration scenarios, the SAML Service Provider Interface (SPI) is provided, as described in GSA Notes from the Field: Security. The SAML SPI is responsible for managing authentication and authorization checks across diverse systems and protocols. This capability provides great flexibility in how security will be implemented. You can:
● ● ●
Purchase pre-built providers (see the Google for Work Solution Marketplace for examples) Build a custom security solution Integrate with an SSO via SAML
The Google Search Appliance also supports the definition of access control lists (ACLs), so that authorization checks can be performed against documents using early binding. ACLs not only enhance performance, but give you more options for managing security. This capability also gives you options to phase your secure search deployment. For information about secure serve and policy ACLs, see GSA Notes from the Field: Security.
10
Once the user ID(s) and groups are gathered, it might be necessary to influence the ordering of the authorization mechanism, for example, it might require that a CMS not allow checking for cached results while it is highly desired on another platform. Rules like these, including the ordering of authorization methods for a pattern, can be assigned with a Flexible Authorization rule and should be carefully considered in complex environments.
Resource allocation requirements Having the right experts available at the time they’re needed is critical for a project delivery as it can drastically delay a project otherwise. It is therefore important to make sure that the right people are assigned to their task when it is required. For example, the SharePoint administrator might be required to support the configuration of the GSA connector for SharePoint or assist when testing the system. Planning the allocation in advance also has the benefit of each person knowing when a specific task will be expected from them, thus allowing them to consider the task in their own respective schedules. Typically, the following experts are required in a project on the GSA owner's side:
● ● ● ● ● ● ● ● ●
Project manager Technical project manager Business owner Business analyst Repository admin Network admin Security expert GSA admin Developer
Typically, the following experts are required in a project on the Google for Work partner side:
● ● ● ●
Project manager GSA experts Repository experts Developers
Performance and scalability requirements Non-Functional Requirements (NFRs) are typically pure technical requirements. In a search solution, the most common NFRs are:
● ●
Performance Scalability
11
Performance Performance requirements typically revolve around how fast the solution returns results, though there may also be requirements around speed of content acquisition. Performance is typically dependent on a number of factors, including:
● ● ● ● ● ●
Security requirements Content type Corpus size The type of queries being executed Network architecture and performance Additional search functions used (for example Query Expansion, Dynamic Navigation, Dynamic Result Clustering,...)
As a rule, if there are specific performance requirements, you should conduct a performance test early in the deployment to determine changes that may need to be made to the solution architecture. Although the Google Search Appliance itself cannot be modified, changes you can incorporate into your planned deployment include:
●
Configuring policy or per-URL-ACLs to improve serve-time security checking.
●
Deploying a proxy to cache where possible for common searches. This change is beneficial only for public (non-secured) content searches.
●
Minimizing network traffic between the Google Search Appliance and content sources. Although this change mostly has an impact on crawl, reduced latency will improve performance of late-binding authorization.
●
Improving perceived performance through responsiveness optimizations (for example, AJAX), and displaying a progress “spinner” to create the perception of responsiveness.
●
Deploying additional search appliances to spread the load. This change reduces the demand on any single search appliance and helps ensure that capacity is not a constraining factor.
See GSA Notes from the Field: Deployment Architectures for further discussion of performancedriven search architecture. Performance requirements should also take crawling and indexing into consideration. Search appliance indexing adds load to your content systems. If there are specific times of the day in which the content systems must not be affected, then you need to understand this so that you can configure search appliance host load schedules accordingly. Furthermore, if the content system is sufficiently strained, or is particularly slow, you might consider content feeds or a connector as an alternative. Besides the chosen method of indexing (Crawling vs Feeding) and the document size, also consider that features like Entity Recognition or Document Preview might have an impact on the performance, as they usually process additional data during indexing.
12
Scalability Scalability requirements typically revolve around the number of queries per second (QPS) or queries per minute (QPM). As with performance, the QPS that the solution supports depends on the security requirements, content type, query type, network performance, and a host of other factors. Google recommends that where scalability requirements exist, you first re-ratify the requirements, ideally with metrics derived from current searches. In many cases, the scalability requirement is not as high as first stated. While search solutions can be designed to support hundreds of queries per second, in practice, this is usually not required. The kind of scalability requirements needed from a search solution are substantially different from those of a transactional system. For more details about designing a search solution for increased scalability, see GSA Notes from the Field: Deployment Architectures. For information about the number of concurrent connections that the Google Search Appliance can accept, see Designing a Search Solution.
Failover / High-Availability High-Availability of the search environment is a standard requirement for many companies. The GSA provides tools like GSA mirroring that make a high-availability solution easy to setup and maintain. However, for more complex environments, you have to make sure that the failover solution is planned accordingly. With regards to the search environment, a failover solution usually addresses two requirements: ● ●
Indexing high-availability Serving high-availability
Indexing High-Availability Indexing high-availability makes sure that the GSA index is kept fresh even if the primary environment cannot serve. While the GSA is a critical device for this setup, other contributors, such as connectors or feeders usually need to be configured for failover too. While indexing high-availability is usually on the wishlist, it is encountered less often in the field given the relatively high level of complexity and resources involved.
13
Serving High-Availability Serving high-availability ensures that users are still receiving results even though one of the applications needed for serving is down. For public serving, in early binding scenarios (policy- and per-URL-ACLs) and most late-binding scenarios (head-requestor), the GSAs mirroring feature is usually sufficient. However there are scenarios when it is not that straight forward: ●
When using groups returned by a connector (e.g., Active Directory or SharePoint)
●
Also in some late-binding scenarios, such as connector-authorization, SAML Bridge or Cookie Cracking, you might want to make sure to have a failover solution too, as users otherwise might not receive search results when the primary connector is down.
For more details about designing a search solution for failover, see GSA Notes from the Field: Deployment Architectures.
Administration and reporting requirements Reporting is an important part of any enterprise application, and search is no exception. In addition to search reporting requirements, identify other reporting requirements. In particular, pay attention to Non- Functional Requirements (NFRs). Requirements to consider include:
●
The analytical technology to be used (for example, Google Analytics, Advanced Search Reporting, or some other third-party tool)
●
Reporting frequency and distribution
●
Other reporting types that may be required (for example, system monitoring or administration events)
Make sure that you understand the business processes that will use these reports. For example, you should understand the use cases for your reporting requirements and make sure that the reporting strategy will deliver on them.
14
Chapter 3 Identifying Phases Most search deployments fall into one of the following categories, listed from simplest to most complex:
●
Specialized deployments focused on delivering a familiar, powerful search experience to customers of an organization’s public or secured externally-facing information
●
Stand-alone search deployments focused on providing general productivity gains to enterprises and making better use of information assets
●
Search deployments driven by a compelling event or larger deployment, such as implementing a new portal, delivering an Enterprise Content Management (ECM) system, or launching a new Information Architecture project
A search deployment typically targets quick wins to deliver a rich search experience to users rapidly, with incremental, iterative delivery of additional value over the life of the search deployment. Business value is derived from the breadth of content over which the search capability is delivered and the usability and effectiveness of the search experience.
Deployment phases The key to successful search deployments is to deliver early and deliver often. Don’t try to do everything at once. Your users will benefit from getting access to the content they want as early as possible. Delivering early means quick wins that can help drive support with your stakeholders and generate excitement and visibility with your users. Phase scope could be defined in terms of:
● ● ● ●
Content sources Security User groups Usability features
Each phase should include an evaluation task, where you explicitly evaluate user satisfaction, and feature requests from users, business units, the IT department, and so on. As always, evaluate feature requests, including risks associated with implementing and not implementing. In general, since each phase is of relatively short duration, you can use most delivery methodologies, ranging from Agile to Life Cycle.
15
If you are using a more classical variety of development methodology, keep in mind that the development phases of a Google search project are relatively short. In this case, you need to make adjustments so you can effectively deliver a quality search experience in a flexible manner. Many of the technologies that you will use, such as Extensible Stylesheet Language Transformations (XSLT stylesheets), OneBox modules, and so on, can be quickly implemented and rapidly adjusted. You need to have flexibility in your approach to prototype rapidly and iterate on deliverables. This section discusses how you can structure your deliverables and project plans to broaden the search footprint and increase the use of your search solution. Each delivery moves your deployment further along the value curve.
Where to start The Google Search Appliance is designed to be rapidly deployed over core content sources. Leveraging open standards and protocols allows rapid integration of content from a variety of sources and implementation of rich usability features, such as Query Suggestions, Dynamic Navigation, UserAdded-Results, and Dynamic Results Clusters. Phases can be as short as a week or two or as long as a month. Google recommends that you structure your program of work to aim for shorter phases, with rapid delivery of iterative functionality, content, or user groups. In many cases, a single rapid delivery phase is all that is required. However, even when your deployment is part of a longer running, comprehensive program of work delivering universal search across all your enterprise assets, you should still structure your phases to deliver quick wins. Before you commence your search deployment, complete the following core tasks, so that your search deployment specialist can get your search appliance up and running as quickly as possible. Before you start:
●
Rack the search appliance.
●
Configure network settings.
●
Inventory your content sources (including document count).
●
Inventory your security systems, focusing on authentication and authorization
●
Configure your network to allow the search appliance access to all content sources, and if required, restrict access to secure areas.
●
Create any user IDs needed by the search appliance to crawl content.
As a Google for Work Partner make sure to use the tools provided in the Product Updates to ensure all information is captured for a successful project.
16
Phasing your activities You might consider phasing your activities as described in the following sections:
● ● ●
Early development Incremental releases Advanced delivery
Early development Delivery items listed in the following table are typically relatively quick and easy to deliver. Consider them as candidates for early development. Many of these could be considered mandatory—a custom front end for example, no matter how simple, should always be a part of the core delivery. Note that the first phase usually involves a critical content source to be indexed (high priority). However, especially if you are not familiar with the GSA, making yourself familiar with the GSA by using a simple to index content source is highly recommended. The benefits are that managing a simple content source first will help you to assess later phases and gain critical basic knowledge with the search system and processes. The following table lists some candidates for early development.
17
Delivery Item
Type
Complexity
Basic HTTP crawl: ● Intranet ● Extranet ● Website ● Wiki ● Web-enabled knowledge bases (for example, Lotus Notes)
Content sources
Low
File system
Content sources
Medium to high
SharePoint sites
Content sources
Medium to high
Basic OneBox modules (for example PeopleSearch)
Content sources
Low to medium
Expert Search
Content sources
Medium
AD Groups Connector
Security
Medium
Lightweight Directory Access Protocol (LDAP) authentication
Security
Low
Kerberos integration
Security
Medium
Query suggestions
Usability
Low to medium*
User-Added Results
Usability
Low
Custom front end
Usability
Low to medium
Advanced search reporting
Usability
Low
Dynamic Navigation
Usability
Low
Entity Recognition
Usability
Low to medium
Document Preview
Usability
Low to medium
Wildcard Search
Usability
Low
Primary system users
User groups
Low
Business owners
User groups
Low
*Depending on whether you use out-of-the-box features or a customized implementation. Complexity may vary depending on your infrastructure and environmental configuration.
18
Incremental releases Delivery items listed in the following table are candidates for incremental release. Consider these items and schedule their deployment according to priority (typically based on volume of content and business criticality), and level of effort. In many cases, you can accelerate delivery by using third-party tools (such as connectors) and getting support from qualified Google for Work partners, who are experienced in Google Search Appliance integration issues. Some of these delivery items (for example, customized advanced search) might require some user feedback before full implementation. The following table lists candidates for incremental releases. Delivery Item
Type
Complexity
Portal content
Content source
Medium to high
Non-web-enabled knowledge bases (for example many Lotus Notes Databases)
Content source
Medium
Content Management Systems
Content source
Low to high
Custom OneBox modules (may be secured)
Content source
Low to high
Custom application content
Content source
Low to high
Additional connectors (FileNet, Livelink, Documentum)
Content source
Low to medium
Customized advanced search
Usability
Medium
Advanced usability features (for example, AJAX-driven user interface)
Usability
Low to medium
Additional users dependent on new content
User groups
Low to medium
Managing several GSAs (Unification / GSA^n)
Architecture
High
19
Advanced delivery Delivery items listed in the following table are candidates for advanced delivery. Advanced delivery candidates might require more time or effort to implement or they might not be required at all. If these items are part of your search deployment, you can implement them in parallel with other deployment tasks. This way, you can get users up and running with core content immediately. In some cases, items are structured data sources that require analysis before understanding how best to integrate into the search experience (for example, Business Intelligence platforms). The following table lists candidates for advanced delivery. Delivery Item
Type
Complexity
Advanced security (including ACLs, Cookie Cracking, Silent AuthN)
Security
Medium to high
Custom SAML SPI provider
Security
Medium to high
Different user names (depending on content source) for one user
Security
High
Trusted Applications
Security
High
Record management systems
Content sources
Medium to high
ERP systems (SAP, Oracle, PeopleSoft)
Content sources
Medium to high
CRM systems (for example, Siebel)
Content sources
Medium to high
Data warehousing/BI platforms
Content sources
Medium to high
Other Line of Business systems
Content sources
Medium to high
High Availability for indexing
Architecture
High
High Availability for serving
Architecture
Medium
20
How long should phases be? In general, phases should last anywhere from a few days to a few weeks. Although work efforts vary and require specific estimates, the duration to complete tasks can be derived from complexity, as listed in the following table. Complexity
Duration
Low
2–8 hours
Medium
1–5 days
High
1–4 weeks
The times in this table are guidelines only and will vary, based on your environment and requirements. Google recommends that you perform an analysis to determine the work effort specific to your deployment. In addition to the work effort, you need to allow enough time to acquire content. Strive for having as much content in the index as possible from targeted content sources. This is not to say that you should wait until you get every possible content source into your search solution, but rather that you should have in the index all of the content from the systems you are incorporating in the current release. It is challenging to predict how quickly the Google Search Appliance will acquire content, as the rate of acquisition is dependent on a number of factors, including:
● ● ● ● ● ●
Network performance Server performance Host load Content type Content size Features (e.g., Document Preview, Entity Recognition)
Google recommends running some tests early in the project life cycle to determine content acquisition speed. Use this information to help you plan accordingly.
21
Chapter 4 Defining Success Criteria Before you commence your project delivery, define what constitutes your success criteria, so that you have a clearly defined set of acceptance criteria. Typical success criteria for search deployments include:
● ● ● ● ●
Perception of new GSA-based solution vs legacy search solution (e.g., A/B testing) User-executed assessments of relevance (for example, user ratings) Security tests (authentication and authorization is working for all secured systems) Breadth of content (volume of content in the index—for example, 95% of content in a system) Breadth of roll out (percentage of users activated)
Define and establish the success criteria together with the business owner, and to be realistic, consider feedback from former search solutions. Also make sure to have an ongoing process of analysis, e.g., after each phase, to adjust where needed early on.
22
