Lesson 3

8. Acquisition Process

Chances are excellent that you, at some point, are going to need to engage a third party to create something for you. Either you're going to want to buy something off the shelf, or you're going to want to engage someone to build something for you. Something for you. I mean, I've had that plenty of times. Please build me 20 computers to this specification,or could you please create this testing tool? We need an automated testing tool to go through a document and pick out certain things, pick out certain words, change the words a certain way, and drop those words into a list and format them a certain way. I mean, so you're probably, at some point or another, going to engage a third party somehow. Let's talk about the things you need to be aware of.

And as an auditor, did they actually do these things? Did they keep these things in mind? Obviously, we all have preferred vendors and we have preferred products. What you have to be careful of is that preferred vendors and preferred products actually are in the best interest of the business. We've certainly in the United States in recent history had cases where at very high levels, at the highest levels, contracts were given, major contracts were given unfairly to preferred vendors for reasons that were not necessarily in the best interests of the organization. And I've certainly been in situations where I had this one staffer. She was a volunteer. If I wanted her to be hired full time as a volunteer, I could just bring her on. I knew her skills, and as a volunteer, she developed all the skills that we needed. But for her to actually be hired, she had to go through a complete process, which I, of course, was not permitted to be part of at all. She had to prove competence in these, these, and these. She had to have it. She had to show her resume. She had to do this and this. I knew that she would be selected because she was, without question, the most qualified person for the job. But I could not guarantee it to her. I said, "Look, you're the only truly qualified person because you've worked in this department." You're the only one who actually knows how we do what we do, and there's nobody from the outside who can possibly have that knowledge or that skill set. But she nonetheless had to go through all those things.

She had to actually produce things from a workstation where everybody had to do the same thing. And so it was a very fair, clear vettingprocess, and other managers were the ones who looked at it and reviewed the whole thing and decided that she was indeed the best candidate, but they didn't go into it with a foregone conclusion. So when you're outsourcing or you're bringing people on board, you've got to make sure that it's in the very best interest of the organization. And as an IS auditor, you've got to make sure that there wasn't just favouritism and nepotism. And I'm just going to give my brother-in-law the contract kind of thing. It's got to be clearly in the best interests of the organisation and the company. So we can use this kind of a checklist here. When we're trying to get a vendor or trying to acquire something, we're trying to buy something. First of all, we can see user feedback testimonials. How good is this product already? Or how good is this vendor already, so we can get recommendations? We'll want to have provisions for competitive bidding. You're not necessarily going to go with the lowest price bid, but you want to make the business case that this is the best value, even if it's not necessarily the lowest price. You'll want to be able to have predetermined criteria so that anybody who is on this sort of management team,who is determining whether or not we're going to go with this vendor or that vendor, there's already clear criteria ahead of time for selecting a bid. Not just because we all like this or like the presentation, but there's clear criteria for choosing one bid over the other. You're also going to want to look at the vendors' financial situation, stability, and whether they'll be around. I have to tell you quite honestly that there are a variety of software vendors out there, major operating system vendors or major network operating system vendors. And there have been plenty of managers who have favoured one over another because even though many people said, well, that vendor's product isn't the best product,so what do I know, they'll probably be around in ten years or so, as for this vendor, even though they have a better product, I don't know that they're going to be around next year to support it. So managers make those kinds of decisions. They're worried about not just the best functionality here, but the long-term support as well. So will the vendor be around? Can they provide the support and the maintenance? And also, what are the considerations for security and the upgrades and controls? I don't want to have to buy new hardware every time you have a minor upgrade. So what are these considerations? It should just be a patch, and then when you have a whole new version, we'll consider whether or not to upgrade our systems. And also, we need to be able to test the system to see if it really performs against the requirements. And also, what about the delivery schedule, the prices, the contract terms?

And there have to be things that if they don't produce for a certain amount of time, there's got to be something in the contract that is either punitive or shift. Something so that if they don't produce as required, that you have recourse in the middle. Not just suing them later. And it's a two-way thing. The contract also has to be enforced and supported all the way through the lifecycle, not just after we sign the contract. And then we have a written report that summarises the analysis and provides a justification of, yes, this is the best possible person, the best possible vendor,the best possible product for our particular need. As an IS auditor, you can use this entire list as a checklist when going through an acquisition, hiring, or outsourcing process. As we evaluate vendors, of course we're looking at how fast is their turnaround, how fast is their response if we call them, how good is their customer service? If we're in trouble, and that's part of the service level agreement, If we're in trouble and we call, we expect a response within X number of hours. How fast does the system react? What's the throughput and the maximum workload of the system? Got a great story? In my home state, there was a product that was developed basically for the state to have a database for its Department of Motor Vehicles, and on a test load of maybe 1000 records, it worked flawlessly. And when it hit production and there were millions of registered vehicle owners, it took a half hour just to find a record. Even with the driver's licence provided, there just wasn't sufficient testing and workload testing and stress testing for the actual environment it was going to go into. It works great in a smaller load set. We had the same thing in Africa. If we had just a couple hundred records, no problem. We got a report, just boom. But if we had 1000 records, it took two weeks. What's the problem? And so you have to make sure that you really can stress test this thing before you're totally signed off and done and happy. And then also, how compatible is it?

How big or compact or whatever is it? How well can we utilise it? What's its capacity for handling workloads and capacity to be expanded and have additional features put on it in case we need more feature sets? So these are all sorts of things we're going to be looking at when we're trying to choose a vendor or a vendor's product. There is a concept called Escrow, and with Escrow, you can have sort of a sense of, "We're not going to pay you until we hit certain milestones." Okay, so give us this and you get paid that.Give us that, you get paid for that. Well, if there's distrust on both sides, then put the money in neutral third party hands. The money is right there. And as soon as you satisfactorily hit the milestone, then they pay you. And we've already put our money up front. And so, this is a way that both sides can trust that payment will be made when the milestones are reached. Another thing is that you can also worry about. Hey, what if they go out of business and they're not done developing this thing for us? We need the source code. So you can also put the code in escrow as well. If you guys just decide you have to walk or you dissolve or you can't somehow complete it, we want to be able to grab at least what you've done so far. So that goes into a neutral third party's hands. So we have code escrow as well as money escrow for the contract management process. Like I said before, even after you sign the contract, you need to enforce it. And the IS auditor needs to see,you know, the contract stipulated this, this,and this was being enforced. And so you want to look at that as well. The contract is not just a sign saying "we're done," or "let's go," but it's going to have stipulations in it that also need to be managed and enforced. So when we're evaluating control mechanisms for systems, the auditor, we need to look at, did they acquire this thing because they truly needed it, and what process did they go through to justify the acquisition or the hiring or the outsourcing or whatever? Did it begin with the business need, and did we verify that we met certain prerequisites or requirements? And did we verify that we started gathering requirements for the infrastructure, for the hardware? We weren't just surprised that, oh, we didn't know, we had to upgrade to 64-bit machines. And did we determine if there were different vendors and what the pros and cons of all the vendors? And what was the final process for choosing one vendor over another? Not necessarily just cost. So the IS auditor is going to want to look at all of these things when we outsource, acquire, or pay a third party to assist us with a project.

9. Testing Process

As an auditor, When this system or software that has been developed is ready to actually be deployed, there are three things in particular you're going to be looking at: configuration management, change management, and release management. Now configuration management is, as the name implies, has this system software or whatever been configured the way it should have been so that it maintains consistent performance, consistent functionality, consistent physical, whatever features as was originally specified, as it's supposed to be like. So has it been configured and is it staying configured to do its job properly with the level of performance and consistency that we expect from its requirements? Then with change management, if anybody wants to change anything, there needs to be a process. Either it can be highly formalised or it can be relatively simple, but you don't just allow people to just change things on the fly. Even if a developer wants to change a line of code, there needs to be a reason why it has been deployed.

So we want to manage change and also release management. You'll find that it becomes easier to, unless it's an emergency and you need to send out a hot fix immediately, to send out releases at regular intervals. So the configuration management process here is a simple checklist we should be looking at. We want to have a configuration management plan so when we actually deploy this thing, how are we going to configure it? We want to have a plan to configure it so we can always stay with that plan. However, we configure this system, this application, so if it drifts off of it, we can always go back to what our expectation is. We want to baseline the code and any related documents. Baseline means you establish the starting point for any bottlenecks or problems. We can then develop and fine-tune it from there. But what's the starting point? We want to analyse and present findings on any configuration control results. So when we configured this, what was the result? And we want to see if there were any findings recorded on that. And of course, we want to see that there werereports that provided the status of the configuration and we want to see if there are release procedures because one of the problems you run into when you have different releases and different versions of minor and major updates is that you'll now have inconsistency in releases and that will comeback to haunt you. I mean, we spent over a year trying to make the data that was in one version. The data in an earlier release would now work with the reports in a later release because the schemas of the databases didn't even line up. So you have to be careful about having different release versions, and if some of these are just being tested and are in beta, you don't want them to go into production. And there have been many, many times when we've had minor differences in versions that did not work. So, like a developer is working on a particular problem. He shuts off a whole bunch of functionality just so he can focus on one thing. He'll get to the other functionality later. And now we're using the version that has the wrong functionality. So we have to have some control over the release process and we'll want to perform any configuration control activities and update the configuration status database. Wherever it is that we're maintaining configuration status, we're going to want to update that with releases. Like I said, you want to control the release process.

And what most software vendors will do is they'll have regular schedules where they'll send out maybe every 18 months a service pack or a major release comes every two years or a minor release comes every quarter or something like that. So there's a clear time. Or all non-critical service packs go out on Tuesday or something like that. So you'll want to have some way of releasing new updates. And when we talk about the concept of a release, these are changes to the software that have been authorised and they're bundled together. So now this is a new version, a major or a minor version. So when we're talking about types of releases, we generally use numbering systems to imply: is it a major version, is it a minor version? And different vendors will have different numbering systems. So like, sometimes people will use odd numbers to show that it is one type of release and even numbers to show that it's like another type. So maybe the odd numbers are beta releases and the even numbers are stable releases. And so here's like an example. The major release is 10/0. A minor release is one of 10 and an emergency one of one. You want to make sure you have some consistency here so that when we look at it, we can tell if it was a major issue, a minor issue, a quick fix, or an emergency. So when you are looking as an auditor at the release management process, here is a list of things we want to be looking at. We want to have agreement on the proposed release and the contents of the release. And there will be all kinds of fights. I'll tell you, people will, competing interests will say, "But we've got to have that functionality next month."

No, no, we really need to fix this functionality and somebody has to make a decision. Look guys, in this next release we're going to have these things, but not those things. As a result, we must reach an agreement on the content of the release as well as the order of the release. When do we release new features, for example, broken down by geographical area, customer business unit, need, and so on? So in some cases, we had provinces where we needed to release something different from another. But the difficulty was that what we ended up with was these inconsistent releases, and now we had databases that didn't work with each other and data that didn't work with each other. And so we were trying to bring everybody back to a baseline. So you have to be careful about geographically based releases. Make sure you have a release schedule. Try to figure out what you're going to need to send out the release. Like in our case, they couldn't just go on the internet and download it. The internet was too expensive; nobody had it. So we actually had to send teams out and have the teams carry the releases physically and send them across the country. Get agreement on who's responsible. You know, a project is going to flounder if you don't have accountability and responsibility. So we have to say who is responsible. Who's accountable for this or that? during the release process. One thing that we did was, as the teams went out, we established like a sort of a command centre and the teams could call in on their cell phones because they couldn't really get internet access and they could call in and report where they were and if they had any issues. Have a backup plan totally You never know what you're going to run into there. Maybe during the install of the update, the power got cut in the middle and now the update has corrupted the operating system because it's incomplete. So you need a backup plan to get your users immediately back to where they were. Or in our case, if the computer wouldn't work, they'd fall back on their paper system for awhile, and then we'll enter the paper when we bring back the repaired computer and then develop a plan for the actual release itself and also a plan for support. So I and other people were standing by the phone as the teams went out there and ran into this problem or that problem. We could talk them through how to deal with it by phone. So these are the things we have to look at in the release management process. Now, occasionally, and this is very disruptive, a business may decide that they need to completely rethink how they do things. They need to re-engineer their business processes. This is like they want to do something extremely radical. We are going to completely do things differently. In our case, we transitioned from a more traditional type of development, more of an aprototype but partly SDLC, to agile development. And so we reengineered our whole development process. And so, organisations might be fundamentally rethinking how they even do their work. They might want to dramatically improve their product and improve customer service.

They might want to cut operational costs or they might want to become a competitor, a player in the market or in the business environment. When they do that, this is a really big thing. So we have to define, well, what is it that you need to reengineer and have a plan for that, a whole project for that, realising that there's going to be whole culture changes here and you really need support from the top. We need to understand the process as it is now and figure out, well, why isn't it good? Or why is it outdated? And then redesign it to make it more effective,more efficient, more competitive, whatever the desire is. And as you implement this, you need to monitor the new process. Is this new process doing any better? You know, I mean, we were promised all kinds of things and now, ultimately, we're not any better off. We're totally different, but we're not any better off. And why is that? Did we not train people? Did we pick the wrong product? Did we release it at the wrong time? You know, why is that? And very often, you have to change culture. And so, this is also very different or very difficult. Usually, if you have to change the culture in a department or in an organization,it helps to physically change things too.

Perhaps other things should be shaken up. If people have like, it could be as simple as the furniture has been rearranged, or we have like, a ceremony or the lighting has changed, and we've got new plans and people are going to take on different duties. We're going to trade duties and cross train. If you shake up someone's world a little bit,it becomes a whole lot easier for them in a new sort of environment to adapt to something new. Then if you throw something new on them, they're still everything else is the same. So that's a kind of a psychological technique that sometimes works. So this whole business process reengineering, did we go through all of these steps? And as the auditor, because this is so monumental and momentous, you have to see if they did that. As an IS auditor, we want to look at these things when we are looking at business process reengineering and evaluating their readiness. Were the stakeholders identified? Were they involved in the decision-making process? Were they consulted for their area of expertise? I've had directors that just decided that they knew best and they would not listen to their senior people, and they made decisions that they didn't even want to hear it.And we all suffered later. So, as managers, did the managers consult those who actually know what's going to happen? Where were the stakeholders and users during the testing process? Did they sign off and say, "Yeah, we're happy, this works fine?" If there were any last minute concerns, I did ask them, "Is this totally working the way you want? Just use it the way you normally do." Are there any last minute things? If there's any change or configuration or releasemanagement processes that were uncovered that might be candidates for business process reengineering, as we're doing this,did we discover, oh, we really should redo how we consider that as well? Did any of those get uncovered and might be a target for the next project or part of this project where all the inputs are in and the outputs are adequately tested? And of course, do we have the appropriate documentation?

10. Information Systems Maintenance Practices

Let's talk about system infrastructure and lifecycle management. Part of your job as the auditor is to see, OK, they had a plan. Did they follow their plan when they were developing software, when they were developing systems, when they were developing infrastructure,and how did they manage to manage their plan? And part of being an IS auditor is also knowing some of the basics of project management. So let's talk about when you are doing systems and infrastructure, lifecycle management, and life cycle. Of course, this includes everything from is this even feasible? to putting it into production and maintaining it, as well as everything in between. We need to, first of all, determine the business case for this. Again, remember, everything that it does should support the business objectives. So what is the business case for this particular software? We need to develop this particular system. We need to put into place this particular site we're going to build. Why are we doing it? What even is the business case for this? And then we need to look and see, OK,what project management framework are we going to use,what governance practises are we going to use? And then, as we have this whole thing going along, we need to be reviewing the project and how it is coming along. We also need to see what control mechanisms are in place. Remember, a control is anything from a policy or procedure to some kind of technical configuration or software to make sure that unauthorised changes don't happen and to make sure that anything that happens we know about and we can mitigate risk. We also need to look at the development and testing processes, and we need to see, okay, what are you doing? How are you actually developing the system or the software and what are you doing to test it along the way to make sure that it is actually going to work at the end? And then, are we ready to roll it out? And then, all right, we're ready to roll it out.

Let's roll it out. Let's migrate to it. However, we're going to deploy it. So when we look at the lifecycle, you start out with just a feasibility study in a large organisation with a feasibility study, you're trying to prove that it's even worth it. To go into this in a large organisation will give somebody a chunk of money, like even $100,000. OK, find out if this multimillion dollar project is even worth doing and what are the pros and cons,what's the feasibility, what is the business value that we hope to get out of it? How long do you think it might take? What kind of resources will be involved? What kind of impact will it have on everything and everybody? So let's start with that feasibility study. From there, we take a look at it and go, "Okay, this looks pretty darn good. All right. Okay, let's try it. Or let's try it in a pilot, or let's try it in a smaller sort of a smaller scenario, or let's just go for it. So we start with a feasibility study to prove that it's worth taking the risk to even try to do this project, whatever it is. The next thing you need to do is gather the requirements. Now, gathering requirements, this is where you involve those business analysts, those system analysts, those businessBSA's essays, whatever you want to call them. With the business analysts, we're trying to find out what it is we need. And when you gather requirements, the Bas will be interviewing the stakeholders, the project sponsor, and the main managers,and that could be anything from phone calls and phone conferences to face-to-face interviews. What is it that you need? And when you're doing the requirements gathering, you're not so much showing what the solution is. You want to know what people need. You want to know what management needs. You want to know what the end users need. Sometimes you'll have a focus group, okay? So you'll have different people. Like, you'll have maybe a salesperson and a department manager and someone who inputs data just to find out their whole take on this idea. Maybe we're going to have some in-house products that will make it easier for us to sell something to our customers. And so then you get the people involved and you start talking with them and you interview them. But when you prepare for this, you have to have all the questions in place and you'll often follow up with additional phone calls or emails to thank them for all this. From the IS auditor's perspective,requirements gathering is tricky.

So many people don't do it well, mainly because the people who want something generally only have the vaguest notion of what they want. Then they certainly have no idea of what it will take. Many clients have seen so many clients have unrealistic expectations or they start immediately having the method, the technology, and the solution. Well, let's find out what you really need. First of all, it takes a while, and it takes a lot of skill, and it takes a lot of talking to people and just finding out stuff. When you've gathered the requirements, the system analyst has to be able to translate those requirements into very concrete use cases that the application developers can actually use. because there's so much opportunity for confusion here. And this is where there's so much opportunity for scope creep. I've been in plenty of things where the client said, "Well, we just need something that is like this and it's already half done." And when I've gotten in there and looked, it's not half-done, it's like this much is done, but the clients thought it was half-done, or only this much is usable. And then they say, "Oh well, not only do we need it like this, but it has to be accompanied by that." Oh, you didn't say that before. Oh, sorry, my bad. So you have to know really ahead of time,and they generally don't know your clients and your customers don't know that in a large organisation, clients and customers are not necessarily outside people.

They are the departments within the company or the business units within the company that the IT department is creating a system or an application for so that the business itself can function. It's so often because that is who your customer is,not outside customers that aren't part of your organization. So, gathering requirements is extremely important. You should get a number of people to do it,and you should really think it out. Okay, what is it? We're going to ask them. When you are auditing this, you need to know what process they went through. And there should be, in a large organisation, some templates people can follow. Ultimately, requirements gathering turns into very clear specifications for the next phase, which is design and planning and design.So when we design, we're actually creating, like, high-level diagrams. What is this thing going to do? We're creating use cases, and they're actually in software development. There are actual things called use cases where you actually draw like little stick figures and say, "This actor needs to do that." So we need to be able to get from the requirements these very concrete specifications, information flows, diagrams, and use cases. After that, we then see, okay, are there any technologies that we're going to select to use to develop this thing, or is there anything off-the-shelf that we can purchase as part of this? And when we go through a selection process, we have to really vet the things we're working with. When I was in Africa and working on a health informatics project, the director of the project had already vetted all kinds of off-the-shelf products and finally decided that none of them suited our needs. We're going to outgrow it. And so you spend a lot of time looking at, okay, what is it that we have already? Can we use any of it? Is there anything off the shelf?

Okay, we have to home grow it. What tools will we use? Then we actually get down to the development of it. And this is where the application developers, if this is software, are actually writing the code and, if it's a system that we're developing, this is where we're actually installing operating systems, loading on applications, loading on whatever tools are in the operating system,or creating virtual machines, or just building whatever thing is during the development. It is really crucial to always check, is this really what we wanted, what they originally wanted? And that's where the system analyst and the developer have to always work together. Is this still where we want it to be? Are we going in the right direction here? As we're developing, we're finally starting to get ready for the rollout and for the deployment. That's when we configure. And of course, we've been testing all the way along, and we have a test plan. As early as our early planning stages of our design, we've developed a test plan and we're testing every step of the way. Now we're finally ready to roll this thing out; to deploy it; to implement it. And usually you'll deploy it in a prototype or rather in a small pilot, and then you'll roll it out in phases. You don't generally just drop it on everybody. You roll it out, check it, do a few tweaks, then roll it out here and then check it, a few tweaks. And then you slowly roll the whole thing out. all the while having a method for rolling back and reverting back to the old system in case things aren't working. When we've rolled it out, you don't just simply throw it over the fence and say, "Okay, maintenance team, your job." Operations is your job. You still have to warrant and support the work. I've been, you know, in projects where the developers write something, then they basically throw the code over the wall and say, "Okay, you guysinstall it; I'm done for the day." And they just don't think about it again. And they don't realise that they need to be there to fix something because something broke. I mean, plenty of times I've been there. We've installed it in a small sort of test and totally broke and broke the system. And now we've got to have the developer figure out what was wrong. And sometimes the things that break are really unexpected and really weird, and then they have to track down what happened. Same thing.

If you're not just installing an application but a whole system or an infrastructure, Whoever developed this thing has got to be ready to support it, warranty it, fix it, tweak it. And then, eventually, the maintenance gets shifted over to whatever operations team is involved in maintaining it. So when we start out, when we have the feasibility study, here is an example where we're trying to describe what the problem is that this thing is trying to fix. What's the scope, what are the risks, and what solutions or actions are we proposing here? Then we write a business case. We say, okay, this is the reason why we want to do it. And this is an opportunity for the business to save X amount of money, to expand in this market, to solve a customer problem, to solve a department problem, to solve a flow problem, to solve something, to improve something, or to expand, improve, or increase something. And so here's our opportunity. Here are the other alternatives. Here's the cost, the benefits, and the various analyses. And then also, when we take a look at this, we go, OK, well, what could the benefits be? And you need to realise that sometimes you don't know the benefits until way later. And for that reason, recognising that sometimes you don't realise that there will be plenty of benefits to something that you just don't realise until a year later, two years later, or six months later. So the benefits realisation themselves must be managed because we don't always know in our plan what the benefit will be. just like we don't always know what the problem will be or what we'll encounter. It might take a while.

So we have to just keep an eye on the benefits and manage the benefits. And when we take a planned approach so that we can look ahead, we can say, when we get farther down the road, we can say, okay,well, based on that plan, what were the benefits? Were there any unexpected benefits that came out of this? As a result, when managing the benefits, you must look beyond the project's immediate life cycle. Perhaps because we completed this project, it influenced other changes or positioned us to be prepared in advance for some environmental change. So these are the things that we need to keep an eye on and be able to manage and be able to prove that,hey, you know, remember that thing that we did two years ago? A few years later, we realised continued benefits. So when you are auditing all of this stuff,make sure they have that feasibility study. What were the results of that? Did they adequately gather requirements? Because poorly gathered requirements or insufficiently gathered requirements will cause all kinds of scopecreep, feature change, confusion, we have to redo things, delays, costs, etc. And then, were we able to write a business case? And then did we translate the requirements well into the design with constant communication between the analysts and the developers? And then did we do testing? Did we plan for testing all the way along? And then how was this thing rolled out? And finally, what benefit did they realise immediately and down the road?

Study with ExamSnap to prepare for Isaca CISA Practice Test Questions and Answers, Study Guide, and a comprehensive Video Training Course. Powered by the popular VCE format, Isaca CISA Certification Exam Dumps compiled by the industry experts to make sure that you get verified answers. Our Product team ensures that our exams provide Isaca CISA Practice Test Questions & Exam Dumps that are up-to-date.