Prerequisites
Before reading this chapter, we advise you to familiarize yourself with SCRUM methodology. Here are two links to compulsory reading material and a short video:
Other interesting resources:
Why use an agile approach for business intelligence projects?
The risk associated with using a V-cycle-type method to conduct a business intelligence project is the belated production of deliverables that no longer correspond to users’ needs.
Frequently mentioned causes of failure in business intelligence projects include a lack of alignment with the company’s strategy, a lack of correspondence to users’ needs, the temptation to create a pointlessly complex, labyrinthine system, and being led by the choice of tool.
Business intelligence projects therefore need make optimal use of the strategies employed in the agile project management approach, which includes being able to adapt to changes in users’ requirements and therefore increase their engagement in the project through timely and more frequent deliveries.
Two schools of thought: the Inmon and Kimball approaches, apply to data modeling in data warehouses, which are central to business intelligence technologies. Kimball’s approach, developed in 1996, was already in phase with agile project management methods. Indeed, the notion of bus architecture adds flexibility to the dimensional data model.
Click the following links to learn more about this subject:
- Adopting an agile approach to decision-support information systems: using agile iterations to create your data warehouse
- Agile Business Intelligence
- SCRUM feedback on a BI project
Introduction to the SCRUM approach applied to Business Intelligence
We have opted to present the SCRUM approach, but what matters most is using an approach that suits your project and your team’s needs. For this course, all groups are required to use SCRUM in order to simplify the teaching arrangements.
Different roles
Product owner
The Product owner is the stakeholders’ and client’s representative. The product owner is not the actual owner of the product but is in charge of defining the product overview and roadmap in order to manage the Product Backlog.
- Clearly expressing the items of the Product Backlog
- Ranking the items in the Product Backlog in order to optimize the achievement of the objectives and missions
- Optimizing the value of the work carried out by the Development team
- Ensuring that the Product Backlog is visible, transparent, and clear to everyone, and that it shows what the Development team will be working on next
- Making sure that the Development Team properly understands the items in the Product Backlog
The Product Owner can personally carry out the above-mentioned tasks or delegate them to the Development team. However, the Product Owner retains overall responsibility for these aspects.
– – The SCRUM Guide - Developed and maintained by Ken Schwaber and Jeff Sutherland
SCRUM Master
The SCRUM Master is responsible for ensuring the correct application of SCRUM methodology and must act as a coach, while informing the Product Owner and the Development team about the approach. However, it should be noted that the Scrum Master is not the project manager, even though the same person may sometimes take on both roles.
The team!
A SCRUM team is a multidisciplinary team composed of around ten members. Its multidisciplinarity relates to the fact that the team members collectively possess all the expertise required for the implementation of a decision-support project, rather than a single person possessing all of these competencies.
In the context of a Business Intelligence project, team members include people closely involved in the business activity, experts in dimensional data modeling, ETL experts (who are in charge of the data preparation and population processes), experts in storage technologies (especially OLAP), and experts in dashboard design and implementation. Given the numerous competencies required, versatility is therefore an asset for members of agile decision-support project development teams.
SCRUM tools
User Stories and your product overview
User Stories correspond to a description of functionalities produced using your client’s terminology. What can complicate business intelligence projects is when clients know what they want: “I want to improve my overview of products in inventory” but are unable to describe this need in a form resembling a User Story. The Product Owner is therefore in charge of supporting the client and the Development team in this process by understanding the client’s activities and proposing User Stories (which may be illustrated by mock-ups of dashboards) designed to meet its management needs.
A User Story consists of the following elements:
- An identifier
- A name
- A short description of a functionality, provided separately from the other stories (if possible):
“In the capacity of …, I can …” - An estimate of the time required for development and testing
- An estimate of the value for the project
- A description of the acceptance tests
User stories focus more on the crux of the problem, meeting the functional requirement. They require us to focus on the matter at hand. User stories have a very clear format; they may be enhanced with certain additional information items, but whatever the circumstances, they require acceptance criteria!
– – Interview with Jean-Pierre Berchez
The Product Backlog
– – The Scrum Guide - Developed and maintained by Ken Schwaber and Jeff Sutherland
The Product Backlog is therefore composed of User Stories but may also contain more conceptual elements which are clear but still lacking in detail. In all cases, the Product Backlogis a list ranked according to the priority and value of the items for the project to be carried out. This list will change over time, as the client can add or remove requirements and the priorities are reviewed at the end of each sprint.
After the production of the Product Backlog comes the important step of estimating the time required to produce user stories. We advise you to use the planning poker technique for this purpose, using this tool, for example.
SCRUM events
The start-up phase
The start-up phase, also known as Sprint 0 allows sufficient time to carry out the preparatory tasks for the project: definition of the User Stories and the initial Product Backlog, in particular.
Depending on the maturity of your client’s definition of the decision-support project, an interview phase is frequently required in order to develop an understanding of the activity and identify the User Stories that will define the client’s management needs.
The following tasks need to be carried out:
- Exploration of the subject, preparation of the user guide and performance of the interview (see the guide to Conducting an interview)
- Analysis of the interview, creation of User Stories, the Product Backlog, and definition of the tasks
The functional or even technical design of a functionality is carried out on the fly in each iteration. The details of the need are only considered in detail during preparations for implementation (as a reminder, the need associated with this functionality was outlined during the upstream estimation phase). This approach has the merits of enabling the client to adjust or further develop its need for functionalities implemented later. It also enables the client (as well as the team) to delay certain decisions that pose no risk to the project (which is often the wise choice when visibility is limited).
– – Florent Lothon, author of L’Agiliste website
Sprint planning meeting
The Product Owner, with the team, must review the product overview, delivery dates, the sprint objective, and the contents of the Product Backlog. The requirements that the teams feels capable of meeting during the sprint should be put at the top of the Product Backlog.
The Development team then lists the tasks to be carried out during the sprint, which constitutes the Sprint Backlog. The design should pay sufficient attention to aligning all of these tasks, which are added to a task table such as a Kanban board (see photograph below).
If certain tasks prove to be impossible to accomplish during the sprint, the team members must amend the list of requirements to be met, in consultation with the Product Owner.
Performance of the sprint
At the start of the sprint, you must produce:
- detailed functional specifications for the sprint in progress, namely:
- the selection of User Stories to be embedded,
- dashboard mock-ups,
- terminological and descriptive information required to clarify any ambiguities.
- detailed technical specifications for the sprint in progress, i.e.:
- BEAM✲ event tables,
- BEAM✲ dimension tables;
- the BEAM✲ event matrix,
- an improved BEAM✲ dimensional schema,
- a specification for the ETL phases by means of a logical datamap and all algorithms to be implemented.
- a test specification with the exact values to be found for several indicators and for a set of values with fixed dimensions.
Progress on the sprint will be monitored via a burndown chart showing changes in the amount of work remaining in relation to the time remaining before the start of the sprint.
– – Florent Lothon, author of L’Agiliste website
A daily meeting lasting no more than 15 minutes should be held in front of the task table in order to enable everyone to review:
- the tasks carried out on the day before,
- the tasks to be carried out today,
- any difficulties that might prevent the attainment of the objectives.
The SCRUM Master should lead this meeting to ensure that all team members are in phase with each other. However, any time-consuming problems should be dealt with outside this meeting.
Sprint Review
– – *Florent Lothon, author of the L’Agiliste website
Useful link:
Sprint retrospective
Useful links:
Questions about the management of decision-support projects
Following your bibliographic research, you should be able to answer the following questions:
- What tasks must be carried out for the end-to-end performance of a functionality in a decision-support project?
- What is the SCRUM approach? What principle is it based on? Which main roles can be identified?
- What are the advantages/disadvantages of an agile approach for a business intelligence project?