Nội dung của bài thi gồm:
Business Outcomes – Sau khi hoàn thành chuỗi kiến thức này, test manager có khả năng:
- Manage a testing project by implementing the mission, goals and testing processes established for the testing organization.
- Organize and lead risk identification and risk analysis sessions and use the results of such sessions for test estimation, planning, monitoring and control.
- Create and implement test plans consistent with organizational policies and test strategies.
- Continuously monitor and control the test activities to achieve project objectives.
- Assess and report relevant and timely test status to project stakeholders.
- Identify skills and resource gaps in their test team and participate in sourcing adequate resources.
- Identify and plan necessary skills development within their test team.
- Propose a business case for test activities which outlines the costs and benefits expected.
- Ensure proper communication within the test team and with other project stakeholders.
- Participate in and lead test process improvement initiatives.
Tài liệu chính ôn luyện cho bài thi:
- Advanced Level Syllabus (2012) Test Manager
- AL 2012 Sample Questions ATM v1.3 Questions
- Advanced Software Testing Vol. 2: Guide to the ISTQB Advanced Certification as an Advanced Test Manager by Rex Black
Khi đọc tài liệu, chúng ta cần để ý đến cấp độ nhận biết (cognitive levels, còn được viết là K-levels)
(http://www.istqb.org/exams-root/what-are-k-levels.html)
- K1: Remember – The candidate should remember or recognize a term or a concept.
- K2: Understand – The candidate should select an explanation for a statement related to the question topic.
- K3: Apply – The candidate should select the correct application of a concept or technique and apply it to a given context.
- K4: Analyze – The candidate can separate information related to a procedure or technique into its constituent parts for better understanding and can distinguish between facts and inferences.
- K5: Evaluate
- K6: Create
Thời gian để ôn luyện:
- Tự học kiến thức: 40 giờ
- Review và làm practice test: 15 giờ
- Có thể tham khảo study plan ở đây
Tổng thời gian cần thiết là 48 giờ tự học và thực hành. Nếu mỗi ngày dành 1 tiếng để tự học thì 1.5 tháng là đủ tự tin để thi chứng chỉ. Nếu có tham gia training và có người hướng dẫn thì thời gian sẽ rút ngắn đi rất nhiều, tầm 1 tháng là thi luôn.
Format bài thi: 65 câu hỏi, trong 3 tiếng
Đặt lịch thi online hoặc offline tại đây: https://isqi.org/en/15-1795694-istqb-certified-tester-advanced-level-test-manager.html
Hoặc tại đây: https://astqb.org/certifications/istqb-advanced-level-test-manager-certification
KIẾN THỨC TỔNG QUÁT
1. TESTING PROCESS
What is Testing Process?
Key words cần nắm rõ: exit criteria, test case, test closure, test condition, test control
test design
test execution
test implementation
test log
test planning
test procedure
test script
test summary report
The test process contains 7 activities:
- Planning, monitoring and control
- Analysis
- Design
- Implementation
- Execution
- Evaluating exit criteria and reporting
- Test closure activities
The activities in the process may overlap or take place concurrently.
1.1.1 – Planning
Test planning starts at the initiation of the test process for each test level and continues throughout the project until the completion of closure activities for that level.
Test planning is also an ongoing activity.
Objectives:
- Identify activities and resources required to meet the mission and objectives identified in the test strategy.
- Identify the methods for gathering and tracking the metrics that will be used to guide the project,
- Determine adherence to plan and assess achievement of the objectives.
During planning stages:
- Determine useful metrics
- Tools can be selected
- Training can be scheduled
- Documentation guidelines can be established
- Select the test strategy (or strategies)
Outputs:
1.1.2 – Monitoring and control
A testing schedule and monitoring framework needs to be established to enable tracking of test work products and resources against the plan. This framework should include the detailed measures and targets that are needed to relate the status of test work products and activities to the plan and strategic objectives.
Test control is an ongoing activity. It involves comparing actual progress against the plan and implementing corrective actions when needed. Test control guides the testing to fulfill the mission, strategies, and objectives, including revisiting the test planning activities as needed. Appropriate reactions to the control data depend on detailed planning information.
1.2. Test Analysis
Test analysis is the activity that defines “what” is to be tested in the form of test conditions. Test conditions can be identified by analysis of the test basis, test objectives, and product risks. They can be viewed as the detailed measures and targets for success (e.g., as part of the exit criteria) and should be traceable back to the test basis and defined strategic objectives, including test objectives and other project or stakeholder criteria for success. Test conditions should also be traceable forward to test designs and other test work products as those work products are created.
Test analysis for a given level of testing can be performed as soon as the basis for testing is established for that level. Formal test techniques and other general analytical techniques (e.g., analytical risk-based strategies and analytical requirements-based strategies) can be used to identify test conditions.
Factors to consider when deciding on the level of detail at which to specify test
conditions, including:
- Level of testing
- Level of detail and quality of the test basis
- System/software complexity
- Project and product risk
- The relationship between the test basis, what is to be tested and how it is to be tested
- Software development lifecycle in use
- Test management tool being utilized
- Level at which test design and other test work products are to be specified and documented
- Skills and knowledge of the test analysts
- The level of maturity of the test process and the organization itself (note that higher maturity
- may require a greater level of detail, or allow a lesser level of detail)
- Availability of other project stakeholders for consultation
Audits
- Audits are usually performed to demonstrate conformance to a defined set of criteria, most likely a process, an applicable standard, regulatory constraint, or a contractual obligation.
- Audits are intended to provide independent evaluation of compliance to processes, regulations, standards,…
Key Characteristics of Audits
- Conducted and moderated by a lead auditor
- Evidence of compliance collected through interviews, witnessing and examining documents
- Documented results include observations, recommendations, corrective actions and a pass/fail assessment.
2. TEST MANAGEMENT
3. REVIEWS
Review within a project include:
- Contractual reviews
- Requirements reviews
- Top level design reviews
- Detailed design reviews
- Code reviews
- Test work product reviews
- Test entry and test exit reviews
- Acceptance review
Review leaders are able to:
- Implement efficient reviews in their projects and demonstrate the benefits of these reviews.
- Ensure than an environment exists is in favor of the implementation of the success factors.
- Should devised a measurement plan to ensure that the reviews provide effective value.
- Make sure all review participants must be committed to the benefits of a well-conducted review.
- Facilitate participants in reviews have review training.
Key characteristics of management review:
- Conducted by or for managers
- Conducted by or for a stakeholder or decision maker
- Check consistency with and deviations from plans
- Check suitability of management procedures
- Assess project risks
- Evaluate impact of actions and ways to measure these impacts
- Produce lists of actions items, issues to be resolved and decision made
Managing Reviews
Project reviews are frequently held for the overall system and may also be necessary for subsystems and individual software elements.
The number of reviews, the type of reviews, the organization of the review, and the people involved all depend on the project size and complexity, and on product risks.
A plan need to be defined with all these types of reviews, this plan should be incorporated into the testing plan which in turn might be incorporated into the overall project plan.
The review plan should answer the questions:
What should be reviewed (product and process)?
Who have the experience to be involved in specific reviews?
Which relevant risk factors needed to be covered by the review?
and many more…
- What to be reviewed?
- Early in the project planning phase, the review leader should identify the items to be reviewed and select the appropriate review type and also decide on the level of formality.
- The objective of the review process must be defined during test planning.
- You might decide that your team need additional review training and add this activity to your plan as well.
- How much to conduct the review?
- A budget which include the cost of the time spent in the review process, the cost of any resources needed by the review process can be allocated.
- Determination of the budget should include a risk evaluation and return on investment computation.
- The return on investment (ROI) for reviews is the difference between the cost of conducting the review, and the cost of dealing with the same defects at a later stage (or missing them altogether) if the review had not been done.
When to perform the review?
- This optimal time depends on the following:
- The availability of the items to review in a sufficiently final format
- The availability of the personnel for the review
- The time when the final version of the item should be available
- The time required for the review process of that specific item
What to measure(metrics)?
Adequate metrics for review evaluation should be defined by the review leader during test planning.
Who should do the reviews?
- Participants in reviews must have the appropriate level of knowledge, both technical and procedural.
- Thoroughness and attention to details are some of other required skills for reviewers.
- Clarity and correct prioritization are attributes to look for in good review comments.
What risks to cover during review planning?
- Review planning should address the risks associated with technical factors, organization factors and people issues when performing reviews.
- The availability of reviewers with sufficient technical knowledge is critical to a successful review.
- All teams within the project should be inloved in planning for the reviews.
- Ensure that each organization is allocating sufficient time for required reviewers to prepare for, and participate in, the reviews at appropriate points in the project schedule.
- The unavailability of the key reviewers due to changes in personal or business plans. In that case, backup reviewers should be identified during review planning.
Metrics for Reviews
- Review leaders must ensure that metrics are available to:
- Evaluate the quality of the reviewed item.
- Evaluate the cost of conducting the review
- Evaluate the overall benefit of having conducted the review.
Metrics for Review Product Evaluation:
- Work-product size
- Preparation time
- Time to conduct the review
- Rework time to fix defects
- Duration of the review process
- Number of defects and their severity
- Defect clusters
- Type of review
- Average defect density
- Estimated residual defects
Metrics for Review Process Evaluation:
- Defect detection effectiveness
- Improvement of review process
- Review progress
- Defect number and severity
- Participant satisfaction
- Cost of quality
- Review effectiveness by type
- Number of reviewers
- Defect detection rate
- Estimated project time saved
Phases of Formal Reviews:
Planning -> Kick-off -> Individual preparation -> Review meeting – > Rework
- Formal reviews have a number of characteristics such as:
- Defined entry and exit criteria
- Checklists to be used by the reviewers
- Deliverables such as reports, evaluation sheets or other review summary sheets
- Metrics for reporting on the review effectiveness, efficiency, and progress
4. DEFECT MANAGEMENT
5. IMPROVING THE TESTING PROCESS
6. TEST TOOL AND AUTOMATION
7. PEOPLE SKILLS
Individual Skills
There are more than one source from where the tester can gain experience and can contribute to the tester’s knowledge base:
• Use of software systems
• Knowledge of the domain or business
• Participation in various phases of the software development process activities including analysis, development and technical support
• Participation in different software testing activities
Aside from ISTQB, if you look at this list, I can say that this what distinguish a normal tester from a super tester.
Super tester look for every opportunity to grow his knowledge.
Normal tester just look at the software at hand and the requirement document and try to prove that the software is working.
Super tester knows more about the business, domain, environment, tools and get involved in any software activity analysis, design, reviews and not just sit waiting for the task to come to his or her desk to work on it.
End-users of software systems have a good understanding of how the system works, where failures
would have the greatest impact, and how the system should react in various situations.
Users with domain expertise know which areas bug are of most importance to the business
and how those areas affect the ability of the business to meet its requirements.
Testers should have the same knowledge as the end users and those users with domain expertise.
This knowledge can be used to help prioritize the testing activities, create realistic test data and test cases, and to verify or create use cases.
Knowledge of the software development process (requirements analysis, architecture, design and coding) gives insight into how errors lead to the introduction of defects, where defects can be detected
and how to prevent the introduction of defects in the first place.
Experience in technical support provides knowledge of the user experience, expectations and usability requirements.
Software development experience is important for the use of test tools that require programming and design expertise, and to participate in static code analysis, code reviews, unit testing, and technically focused integration testing.
Specific software testing skills such as the ability to analyze a specification, participate in risk analysis, design test cases, and the diligence for running tests and recording the results.
Technical skills are not enough for the testers to succeed in their career, but they also need to possess and employ the necessary soft skills.
Test Managers on the other hand, should have in addition to the capabilities discussed in the Foundation syllabus and this syllabus, the knowledge, skills and experience in project management since test management includes many project management activities,
For example, making a plan, tracking progress and reporting to stakeholders.
In the absence of a project manager, the Test Manager may take on the role of both the
Test Manager and the project manager, particularly during the later stages of a project.
Beside the managerial skills, the test manager need to possess personal skills and interpersonal skills.
Personal skills is where the successful test professional must be well-organized, attentive to detail and possess strong written and verbal communication skills. The test manager should also show professional
interpersonal skills, such as giving and receiving constructive criticism, influencing, and negotiating,
which are all important in the role of testing.
Test Managers on the other hand, should have in addition to the capabilities discussed
in the Foundation syllabus and this syllabus, the knowledge, skills and experience in project management since test management includes many project management activities,
For example, making a plan, tracking progress and reporting to stakeholders. In the absence of a project manager, the Test Manager may take on the role of both the Test Manager and the project manager, particularly during the later stages of a project.
Beside the managerial skills, the test manager need to possess personal skills and interpersonal skills.
attentive to detail and possess strong written and verbal communication skills.
The test manager should also show professional interpersonal skills, such as giving and receiving
constructive criticism, influencing, and negotiating, which are all important in the role of testing.
In addition to working effectively with others.
Team Skills
A strong team can be built by balancing the strengths and weaknesses of the individuals.
Starting with the weaknesses that have the greatest impact on the team’s effectiveness
and efficiency, the Test Manager should decide how to address those areas.
Approach could be :
• training, For example, send people to a training course, have training sessions in-house, develop custom training, use e-learning courses like this one.
• Another approach is self-study, For example, books, webinars, Internet resources.
• Yet another approach is cross-training, the team members should have a willingness and ability to teach and learn new skills from their peers.
For example, assigning someone with an interest in learning a skill to work with someone who already has the skill on a task requiring that skill, having local experts give brief presentations on their area of expertise, etc.
• (Mentoring is a similar approach, where a person who is new to a role is paired with a senior person who has had that role, and the senior person acts as an ongoing resource to provide advice and assistance to the jenior person.)
In addition to addressing weakness, the Test Manager should remember to leverage the strengths
identified in the skills assessment as part of the training and skills development plan.
Test Team Dynamics
Motivation
Communication
PRACTICE TESTS
REFERENCES
https://www.istqb.guru/istqb-advanced-level-test-manager-dumps-1/
https://killexams.com/software-testing
https://www.processexam.com/istqb/istqb-certified-tester-advanced-level-test-manager-ctal-tm
https://www.justcerts.com/istqb/atm-practice-questions.html
https://www.justcerts.com/bcs/tm12-practice-questions.html
https://www.justcerts.com/istqb/istqb-test-manager-certification-exams.html
https://www.testpreptraining.com/advanced-level-test-manager-atm-istqb-free-practice-test