Economic Revitalization thru a New Software Engineering Approach

This blog is about the use of a new software engineering approach that will facilitate the strategic change that will revitalize the economy. Strategic change is the response by government and business to the strategic factors inherent in the modern business world. The strategic factors include competition, business mergers and acquisitions, trade blocs, globalization, and market and economic conditions. It also includes advances in computers such as hand-held, portable, desktop, cluster or cloud computing. The new software engineering approaches such as organic software engineering that supports social requirements, computer-supported collaboration, and social computing are also included. Finally, new telecommunications such as optical fiber, high frequency, and associated switches and routers are included.

Turbulent, dynamic, uncertain and constantly changing economic environments require business organizations to constantly monitor their environment, reassess their objectives and goals, and then to adapt to the situational context. Strategic change includes the scanning the business environment, analyzing the specific environmental contexts, and establishing organizational purpose. It also includes the realignment the subsystems (job engineering, human resource planning, organizational design, and management) in order to achieve purpose, mission, and vision. Strategic change is needed because many companies have not adapted to global realities. In order to remain competitive, business has to examine these old ways of doing things, identify the conceptual framework that underlie these systems, and ascertain the effectiveness and efficiency of these systems in the new global reality. The world has become asymmetrical and the new global reality often no longer fit the current causal, linear, operational realities. The global winner will be the business that uses information technology to support the redesign, reinvention, and reengineer of their business units. Although American companies cannot compete from low labor costs but can compete on the basis of better quality, time to market, localization, flexibility, adaptability, innovation, agility, and quality.

Reorganization is very important because the strategic factors are forcing often disparate internal and external organizations to realign their business structures in order to work more closely together. The current stovepipe traditional formal organizations generally lack the capacity for strategic change because these structures present serious barriers to the collaboration, cooperation, communication, and coordination required by integrated systems. The closed system, vertically designed business structures where information is controlled from the top-down does not function well in horizontally, integrated systems that must cross organizational boundaries. A more open system organizational design is needed because the interactions, relationships and exchanges between business units are changing and growing even more dynamic, uncertain, and complex. Businesses must act more like biological living systems rather than machines bureaucracies because organic organizations are more adaptive, flexible, and less bureaucratic.

Software technology is the means by which organizations can become organic. As software has increased the value of physical products from cars to cameras, it can also help business unit organize and better compete. New and emerging organizational designs, technologies, and a software engineering approach can help get high quality products, services, information, and knowledge to the customers in a timely manner. It can also integrate and align their financial, managerial, job engineering, human resource planning and organizational subsystems with the situational contexts and business purpose. Examples of key design attributes of innovative organizational designs include:

• Deliberative participative planning through implementation

• Centralized planning but distributed execution and accountability

• Shared responsibility and authority among all members

• System continuous innovations – all subsystems involved

• Flexible, porous, adaptive, response, fleet of foot designs

• Self-directed, self-managed, automated work teams

• Creative, innovative, entrepreneurial, empowered teams and members

• Knowledge sharing, continuous, generative, appreciative learning

• Focused high-performance commitment thinking “outside the box”

• Continuous communication with front line people cross functional, distributed, centralized hierarchical

• Crossing, functional boundaries, “boundarylessness”, “stretch”

• Resistance of stagnation and decline through change, challenge, growth, and opportunity

For example, consider a centralized planning but distributed execution and accountability
design attribute. This is much like the Organic Software Engineering Approach (OSEA). In this form of project design, strategic planning and tactical planning are centralized and the operation planning and software development are distributed. Large projects operate as an ecosystem with three basic communities, the business, the technical, and the external partner. The business community consists of user management teams, customer teams, and user teams. The technical community consists of project management, software engineers and support, and developer teams. The partners are the vendors, prime contractors, consultants, and other organizations that are outside the company. The ecosystem also consists of cross functional teams from various communities and neighborhoods.

The user management team develops the business strategies. The customer teams define the tactical software requirements in terms of the high-level software functional elements such as the systems, subsystem, software functions and capabilities. The project management team identifies the development strategy while the software engineering team develops the tactical development plans that include the technologies, processes, and artifacts (methods, tools, techniques, procedures, models) that will be used to develop the software system. The project shifts from being centralized to distributed when the user/developer teams are assigned their respective subsystems to develop. The integrated user/developer teams will have autonomy at the point of action and control. These teams will identify, prioritize, and schedule the features for each release. At this point the feature requirements are detailed, estimated, planed, assigned, scheduled, tracked, controlled and the software created.

Compare this approach with the traditional software development approach where a project starts with the developers rushing pell mell into the requirements phase. Once the requirement specification document is completed, the SDLC phases, activities, and tasks are defined, estimated, planed, assigned, scheduled, tracked, and controlled in a monolithic manner. The sole obsession is to keep the project on schedules. While the ecosystem concept puts people and the business units in the forefront of the software development process, the SDLC puts activities and schedules in the forefront. In the SDLC developers are like machines whose purpose is use the detailed processes, methodologies, and procedures much like on an assembly line to develop the software.

Information Technology and Strategic Change

Information technology is required to support the innovative designs attributes that will facilitate strategic change. Enterprises are using advances in information technology to help business units respond to the strategic factors in an organic manner. The differences between the machine systems and organic systems represent a fundamental redefinition of the software ideology. An organic ideology calls for open system projects that maximize the relationship with its environment. Organic projects are designed to achieve what the business customers want, high quality software systems that support strategic change, competition, business purpose, customer value, quality, productivity, innovation, and user satisfaction. Organic systems rely on people and the principles and practices of concurrent engineering, systems engineering, and quality engineering. Organic design structures projects and systems like biological organisms. Generally, biological systems contain vertical, multi-level relationships in a system hierarchy. Think in terms of cardiovascular, endocrine, digestive, lymphatic and respiratory systems. Biological systems are self-organized systems where each system supports a body functions. For example, the respiratory system is the system of organs in the body responsible for the intake of oxygen and the expiration of carbon dioxide. In mammals, it consists of the lungs, bronchi, bronchioles, trachea, diaphragm, and nerve supply.

IT helps business units to realign their business structures for greater integration between the various interlocking business groups. Different business units can be organized in different ways according to their situational context and business objectives. A strong convergence of various technologies enables business organizations to engage in diverse, complex, internal and external activities. It allows the business units to keep pace with the changes in technology, demographics, and economic conditions in a turbulent competitive environment. Without information technology, obtaining the necessary structural support among disparate customer organizational units is challenging. Consider the difference in an organization that supports with centralized strategic and tactical planning with flexible, adaptive, and distributed with creative, innovative, and empowered user/developer teams with centralized, hierarchical workers performing manual, operational work.

Software Total Factor Productivity

Software Total Factor Productivity (STFP) represents the productivity gains that business customers derive from creating more outputs with the same or fewer capital and labor costs. Systems that allow the customer to reduce inputs in terms of labor hours and business costs while increasing customer outputs increase productivity. Business growth creates more jobs as the increases in efficiency and effectiveness enables the company to expand its operations. As the success of a business generally depends on its ability to deliver more real value for consumers without using more labor, capital or other inputs, the measure of success of software system is the ability to help the business users increase output value without having to use more labor, capital or other inputs.

The success of software project includes the ability to deliver customer value. Customer value is the advantages that business accrues from a software system. It is the reason why companies initiate software projects. According to Robert Sessions, the loss of business value due to project and system failure is 6 trillion dollars worldwide and 1.22 trillion dollars USA. Sessions gave as an example the new electronic fraud detection system that the IRS spent $185 million to develop. The project was abandoned in 2006. According to a 2008 report by the Treasury Inspector General for Tax Administration, the Federal Government lost approximately $894 million in fraudulent refunds during 2006 because the system was not operational. That was just for 2006 alone, there is always the following years.

Realized systems often provide marginal or insufficient business value. This happens when system objectives, purpose, designs, and requirements are misaligned with the needs of the business. When the system is delivered much of the feature set goes unused because the features are not required, the users are overwhelmed with too much functionality, or the quality of the system is so poor that the users are afraid to explore beyond absolutely what is needed. A poor defect prone system that is hard to use along with inadequate software support systems could increase user dissatisfaction and decrease customer productivity. Finally, business organizations often require more than just a software solution. Many projects require the ability to integrate complimentary business solutions including organizational design, business process improvements, and reengineering under the same project umbrella.
Software Engineering Approach

A new software engineering approach based upon systems thinking must replace the mechanistic, linear, reductionism attributes of the traditional software development approach. A machine thinking ideology like the current SDLC calls for closed system projects that focus on internal needs and minimize relationships with the business environment. Closed systems projects can’t support the achievement of business value, productivity, customer satisfaction and strategic change because that is not their underlying purpose. In closed systems, software engineers are trained, encouraged, and rewarded to deliver requirements specifications on time and within budget. The results are, at best, “technically correct” systems that provide limited business or productivity value.

The SDLC is the software incarnation of the Taylorism. Even the Agile methodologies are based upon software production technique that borrows from lean manufacturing and emphasizes closed systems, efficiency, technology and process. The SDLC supports centralized, hierarchic organizations that rely on the use of detailed standard process, detailed methodologies, and detailed procedures. It ignores the importance of human behaviors and relies solely on technology and process in order to maximize efficiency. Frederick Taylor wrote that manual workers are considered to be like machines and solely motivated by money. He specifically exempted mental workers although mental workers like teachers and software engineers are treated like manual laborers. Software engineers should not be treated like machines because people, how they are managed, their skills, experiences, and knowledge, and their cultures are essential to software success. Large projects require very high levels of coordination, communication, cooperation and collaboration, the 4Cs. People their interactions and relationships are required in order to identify the right objectives, invent the right requirements, and design, develop, and support the right system. They are needed to adapt to various situational contexts, develop the strategies to achieve objectives, develop the tactical plans needed to the concurrently engineer the project, and create and execute the operational plans needed to realize the system.

Organic System Requirements

Joe Goguen, late computer science professor at UCSD identified the requirements gap. He once stated, “It is not quite accurate to say that requirements are in the minds of clients; it would be more accurate to say that they are in the social system of the client organization. They have to be invented, not captured or elicited, and that invention has to be a cooperative venture involving the client, the users, and the developers. The difficulties are mainly social, political, and cultural and not technical.”

Organic systems include the business organization’s social requirements while SDLC, ignores social requirements. SDLC requirements are the raw materials to a deterministic, manufacturing-like software engineering approach that creates poor quality functional tools. Organic software requirements consist of the functional, non-functional, and social requirements that are inputs to an evolutionary, biological oriented approach that create purposeful systems that are aligned with the business units. Purposeful systems are adaptive systems that select the most effective strategies, tactics, and operations based upon the situational context and objectives. Purposeful systems not only include the organizations work tasks and activities, it also includes the managerial, organizational, technical support, human resources and cultural elements. In inventing the organic requirements and system designs, the requirements engineer must conceptually understand the customer organization’s underlying mental model, philosophy, concepts, theories, and principles. They must understand the management constructs, cultural construct, and the organizational construct. The organizational construct includes information about the design structures and team structures, human resources and reward system, job engineering and design, and information and decision making. The cultural construct include cultural behaviors, attitudes, beliefs, norms, and values.

Purposeful systems require the definition of the purpose object as the design principle and unifying element that glue the systems objects and business units together. The purpose objects consist of the business unit’s mission, vision, and values. System definition modeling define “things” like organizations, projects, activities and many other as systems that consists of four subsystems each with eight objects. The objects are purpose, environment, boundary, inputs, processes, outputs, restrictions and feedback control. Mission is the reason why the organization exists. A mission defines something to be accomplished in terms of desired results, objectives, and goals. Values define the preferred behaviors and what is most important to the business organization in terms of thinking, feeling, saying, and doing. Values are defined, developed, and achieved through commitment, communications, and discipline. The vision is the selected outcomes that produce the desired results. Establishing what's in the vision dictates knowledge of the desired outcomes, the current outcomes, and the gap between the two. Consider a current outcome as an organization with centralized planning and control, hierarchic bureaucracy, and process control. Consider the desired outcome as the vision of a distributed, flexible, adaptive organization with creative, innovative, and empowered teams. Define the gap between the two as the social requirements for an innovative design attribute.

Supporting the new and innovative design attributes is not simply a matter of reorganizing an organization. Software systems that support innovative project design attributes require the social requirements that include different communication channels, cross-functional processes, human resources roles, and the data flows. Social system requirements must support and encourage good relationships through improvements in the quality of the interactions between the various cultures, groups, and roles that interact. As technical and process functionality are important to the work context, the ability to communicate, coordinate, collaborate, and cooperates is just as important. As such, there are need for social requirements that support collaboration, communication, cooperation, and coordination. Feedback is also a prerequisite for purposeful systems. In this case the social requirements might align the visual representation of the user interface with the desired behaviors. Studies show that the presentation of performance feedback designed in the user interface can radically alter a decision-maker’s responses and behaviors.

Consider the difference in project management. SDLC project management stresses efficiency through the planning and control of the resources needed to ensure that a project is finished on time and within budget. Project management is essentially task management that includes the creation of complex task networks that use sophisticated planning and tracking software with Pert Charts, critical paths, and earned values. Under the OSEA, however, project management has a more social orientation. With distributed project teams managing their tasks and creating the features, project managers can devote attention to boundary management. Boundary management includes open systems planning, leading the project ecosystem, and maintaining business managers, customer, and users relationships. It also includes clarifying purpose, developing strategies, and maintaining negative and positive feedback channels. Finally it means coordinating technical management, business management and external partner management, leading the development community, and responding to a changing environmental needs and wants.

Organic projects require new software requirements and system design skills, knowledge, and experiences. In addition to the technical specialists, organic projects need client facing generalists that perform activities like social network analysis, cognitive task analysis, software ethnography, organizational assessment and design, and reengineering. While there are some software engineers that require STEM knowledge the vast majority of IT software developers do not. The new roles and skill sets will combine technical knowledge with a background in the organizational, managerial, and behavioral sciences. People will need knowledge and skills in organizational design, quality and productivity, human resources, software ethnographers, organizational psychologists, cognitive software engineers, business and domain specialists, and systems theorists. This will result in the creation of more software jobs because the decrease in development costs will increase new projects and thus create a demand for both technical specialists and generalists.

Summary

“An important question that has been debated for almost a decade is whether computers contribute to productivity growth. Productivity isn't everything. However, as noted by the economist Paul Krugman, in the long run it is almost everything. Productivity growth determines our living standards and the wealth of nations. This is because the amount a nation can consume is ultimately closely tied to what it produces. By the same token, the success of a business generally depends on its ability to deliver more real value for consumers without using more labor, capital or other inputs.” Erik Brynjolfsson of the MIT Sloan School of Management

The question rather computers contribute in ongoing only because of the SDLC. If the SDLC adds value and subsequently increases productivity that that is an unintended consequence. According to Wikipedia adding value requires coordination and collaboration; changes in organizational processes, committed leadership; flexible jobs and adaptable, capable employees, supportive organizational culture and attitudes, and investment in information technology. However, the benefits from investments in information technology have been marginal and create concerns about computer productivity. The article proves that the OSEA will address these issues.

The End of Offshore Development

The offshore of software jobs could in the long term decimate the American IT industry and reduce American software competitiveness. Software is vital because it fuels the U.S. economy and serves as a catalyst for innovation, communication, and potential economic growth. However, over the last decade, American corporations, citing cheap labor costs transferred 2.4 million jobs to India. Concurrently, since 2002, the number of students graduating with computer science and related degrees has declined 58.5%. A 2006 study on “Globalization and Off-shoring of Software” by the ACM indicated that because of off-shoring, there is no way of ensuring lifetime employment for American software engineers. Who would want to study, graduate, and enter a profession with no guarantee of lifetime employment?  The research firm Gartner Inc. predicts that up to 15 percent of tech workers will drop out of the profession by 2010, not including those who retire or die. According Business Week “The U.S. high-technology industry lost 115,800 jobs in 2010, a 2 percent decline that marked the industry’s second consecutive year of falling employment.” The situation, however, is different in India, particularly among the three largest IT companies. Tata Consultancy Services Ltd., the largest Indian IT-service provider, currently with over 202,000 employees plans to hire a total of 60,000 workers this year. Infosys Ltd, the second largest over 133,000 employees plan to hire 45,000, while Wipro the third largest with over 122,000 workers plans to increase from 25 to 35%, 30,000 to 40,000 respectively. Since there is no domestic industry for India IT to support, the vast majority of these are American based.

The result is a diminishing labor pool of software engineers and the end of American software dominance. As the IT industry contracts, American developers limited to Aerospace work and beset by the high costs of software development and poor quality problems will face survival problems similar to the American automobile and commercial electronic workers in the 1980s. Eventually software research and development as well as innovation will follow the jobs.  The DOD will be particularly hard hit because of its reliance on software, the risk of offshore development, and a dwindling pool of citizen software talent eligible for security clearances.  

Software Development Life Cycle

The Software Development Life Cycle (SDLC) that underpins current software engineering practices and methodologies is the reason for offshore development. Simply stated, it does not work for software engineering. It is the core software development problem because it applies a manufacturing, assembly line, manual work, and linear oriented approach to software development. Software requirements are so volatile that a mechanistic, monolithic process that focuses on task management and ignores human issues and behaviors cannot work. Ignored are the human social dynamics that include people, their needs, desires, and fears as well as the impact upon project behavior, motivation, and morale. Projects are more than a sophisticated planning and control systems; projects require collaborating, communications, coordination and cooperating between people. It is the behavioral aspects of software development that make large software projects as much human systems than technical systems. As the late Watts Humphries once stated, software developers resent being treated like grunge. The assumption that human behavior can be ignored and software engineers treated like manual laborers create project performance problems and reduce staff productivity.

Organic Software Engineering

The process of software development and evolution is an ambitious undertaking involving complex, incomplete, sometimes inconsistent and often fuzzy factors.  Variables concerning design, quality, reliability, stakeholder interests and objectives, moving targets, and constraints such as budget and timeline must all be considered throughout a dynamic life cycle. The challenge is to provide sound methodological support for enabling good decisions about processes and products, risks and bottlenecks as well as for selection of tools, methods and techniques.  A need also exists to certify critical software systems to ensure their dependability, relying on evaluation of the software development process as well as the properties of the system.  Software engineering research topics will identify and develop innovative means of meeting the challenge. TECHNOLOGIES OF INTEREST – DARPA 

Closed system software development approaches like the SDLC cause software projects to operate like machines. The requirements phase cranks up the project machine that activates a series of sequential phases, activities, and tasks until the system is implemented. It uses a common solution approach that includes detailed processes and methodologies. The problem is that the machine response is too inflexible to handle DARPA’s accurate description of software reality. How could a machine deal with the inconsistencies, fuzzy factors, uncontrollable variables, performance factors and changing targets?

Open systems such as biological and social systems however operate quite differently from mechanistic systems. Organic systems are designed for the inconsistencies, fuzzy factors, uncontrollable variables, performance factors and changing targets. Instead of a common solution approach, there is the concept of equifinality. All software projects are unique because each project has a set of different system objects including purpose, environment, boundary, inputs, outputs, process, restrictions, and feedback control. It suggests that similar results may be achieved from different initial conditions and in from many different ways. In software development, the implication is that software projects may achieve project system objectives from different situational contexts. Instead of cranking up the monolithic project machine, biological projects define their objectives and goals, organize the project ecosystem, evaluate their situational context, develop the strategies, identify the quality needs, select the tactics, and create the operational plans that when executed will deliver the software in stages of systems growth

The Organic Software Engineering Approach (OSEA) solves the problems associated with the almost infinite number of project variations that make each software projects unique. Software projects are like the Tower of Babel. There are the multitudes of technologies and development methods, tools, techniques, approaches and processes. Software projects are also rife with context variations, unknowns, initial starting conditions, restrictions, constraints, and controllable and uncontrollable variables. The OSEA consists of a perception, paradigm, conceptual framework, behavioral model, and a software development practice called the Organic Software Development Practice (OSDP). The OSDP replaces the mechanistic characteristic associated with the SDLC with biological characteristics associated with living systems. Software projects are not simple, causal, linear systems where the requirements are to simply define the tasks, estimate the effort and duration, and then schedule and monitor the progress. Software projects are complex adaptive systems that fit the above DARPA description of software reality. Software projects that share biological characteristics best handle the uncertainty and need for adaptation that is associated with software development. These systems require a corporate memory in terms of history data and feedback mechanisms that allow a project to learn, grow, survive and be successful. Some would call this organizational learning.

The OSDP consists of two components, the Organic Software Engineering Life Cycle (OSELC) and the Organic Software Engineering Environment (OSSEE). While the OSELC provides the universal system processes that are applicable to all software projects, the OSSEE provides the infrastructure, paradigm, models, and methodologies that allow adaptation to any type of software project situation. The OSELC replaces the SDLC. While the latter structure projects around the tasks that must be performed, the OSELC structure projects around the system processes that underlie software projects.

The OSELC system processes include systems engineering, concurrent engineering, software functional elements, organic design, quality engineering, and joint optimization. Software systems engineering defines how complex software engineering projects should be designed and managed. Concurrent engineering promotes the concurrent development of project plans and system products. It manages the entire software system engineering lifecycle though parallelization, participation of all the constituents in the ecosystems, and an emphasis on business value, customer satisfaction, productivity, and quality. Software functional elements are the new software functional model whose parts consist of systems, subsystems, software functions, capabilities, and features. Organic design partition projects, systems, and software functional elements like biological organisms that support distinct body functions. Think in terms of cardiovascular, endocrine, digestive, lymphatic and respiratory systems. Quality engineering embeds the required quality elements into the software functional elements. Joint optimization balances efficiency in terms of time, costs with effectiveness in terms of business value, customer satisfaction, and productivity.

The OSSEE provides the capabilities to assess, design, and manage the project as a socio-technical system. The Software Paradigm Model (SPM) enables a project to assess, design and manage its social system, including conceptual, managerial, organizational, and cultural constructs. Cultures in terms of attitudes, values, norms, beliefs, customs, practices, and social behavior have a significant impact on project performance. Cultures can be high performance where practitioners thrive on achievement and success or cultures can be negative like the schedule-first cultures that implement immature system when projects fall too far behind schedule. Management has a very significant impact of the project culture. So do the organizational factors that include job engineering, people, information and decision-making, project structure and the reward systems.

The system methodologies determine the manner in which software projects will plan, organized, managed, developed, and realized software systems. The system methodologies differ from the traditional methodologies in that the latter are very detailed and designed for strict adherence. System methodologies offer a set of guidelines, principles, and practices that can be tailored to a specific situational context. The OSSEE methodologies are strategic project management, project ecosystem design, tactical project management, functional element management, quality engineering, and distributed team project management. The system methodologies allow the design of a project, definition of strategic objectives and strategies, development of the tactical plans, management the software functional elements, embedment of quality, and creation of the software services. The software developers and users are assigned to distributed teams that are tasked to develop high-level software functional element like a subsystem. These integrated teams identify and prioritize the capabilities and features needed to support their business processes and then develop the operational plans needed to create the software. These practices are not ad-hoc, at project time, the systems engineer selects from the capability libraries the processes, techniques, methods, and tools that best fit the needs of the technical, project management, human resource, and organization subsystems.

 The Cause of Offshore Development

The traditional software development practices ignore the human aspects of software engineering, particularly the interactions between developers and the customers. Generally, IT staff is not the loyal “company” people that focus on the needs of the business units that they support. The problem is the closed system, ivory tower, mental model that keeps software developers aloof and disengaged from the business units.  Software engineers are trained, encouraged, and rewarded to deliver requirements specifications on time and within budget. Developers attempt to regulate requirements changes in order to meet schedules and when project fall too far behind schedule they will implement immature systems. However, business customers want high quality software systems that meet business purpose, add customer value, satisfy users, and increase quality, productivity and innovation. These differences have hardened the attitudes of many business units and executives against IT organizations. More often than not, software projects are characterized by negative interactions including finger pointing, lack of support and user involvement, conflicts, excessive politics, personnel resistance, and lack of cooperation and communication. These conflict severely affects the quality of the delivered systems.

It requires no stretch of the imagination to realize that if project developers and the business customers could improve their relationships, the results would better. The quality of the interaction and the nature of the relationships are more important to the development of quality, high performance systems than discipline processes. How well people interact with each other is essential to creating business value. People interact in order to identify the business problems, solutions, needs, project objectives, and business requirements. People also interact in order to determine and analyze the software requirements, create the designs, code the software, and test the systems. People interact in order to deploy, support, maintain, operate and train other people how to use the system.

The rancor between internal business units and internal software organizations has made it easy to offshore software engineering jobs. Business units are reluctant to pay the high American salaries for results that do not significantly improve the business. Both inshore and offshore development projects use the same problem-prone, archaic development approaches; subsequently the results from American developers and their offshore counterparts are about the same. Viewed by senior executives as costs centers that fail to produce significant business value, the decision to obtain the cheapest IT job rate, via H-1B and offshore development, is both business and social driven.   

Offshore Incompatibility

The open systems oriented OSEA provides the innovation that will significantly improve the business, technical and human-social aspects of software development. It provides business, developer, and economic needs and wants. The business units receive high quality systems that meet business purpose, add customer value, and satisfy customers through increases in system quality, productivity and innovation. Developer receives reductions in software costs by almost a factor of 10, decreases in cycle time, and attainment of schedules and budgets. The economy will see the end of the productivity paradox, the return of American-based software jobs, and a revitalized economic system. Software will become even more a significant part the value chain than enables American businesses to successful compete despite higher labor costs. As software has increased the value of physical products from cars to cameras, it can also increase the value of business services.

The OSEA is incompatible with offshore development because offshore software companies cannot support the constantly high level of interactive and relationship needs including collaboration, communication, coordination, and collaboration. High interactions are required in order to identify the right objectives, invent the right requirements, and design the right system. High interactions are needed to adapt to various situational contexts, develop the strategies to achieve objectives, develop the tactical plans needed to the concurrently engineer the project, and create and execute the operational plans needed to realize the system. High interactions are needed to receive the feedback, measure against objectives, identify the meaning of any variances, and select solutions.

Although, the American software developer cannot compete with offshore companies on a basis of labor costs, the OSEA allow them to compete on the basis of overall costs, efficiency, effectiveness, quality, time to market, and productivity. The American software engineer now has the winning home court advantage because in order to improve business value, customer satisfaction, and productivity business and developers teams must meet “eye to eye to help see things eye to eye”. Offshore developers and their H1-B counterparts will have difficult time teaming with the business managers, customers, and users because of cultural, accents, and skill nuances. Client-facing generalists with knowledge of the organizational science, behavioral sciences and system theory will replace some of the back-end specialists. Companies will have to hire American workers for customer facing jobs creating even more jobs for Americans including veterans.

There are internal risks in not changing our approach to software development, however. The Washington Time reported that “China is stepping up its long-running struggle with the United States over access to technology. China this year embarked on a campaign to target advanced industries such as aerospace, medicine and information technology for its next stage of development”. However, the Chinese may not share the affliction of many American software engineers. Some American software engineers ignore the need for change although the vast majority of software projects fail to add the business value, customer satisfaction and economic productivity. Like Keebler Elves, they happily pursue their goals of meeting schedules on time and within budget despite the constant failures. Many will never change their view because they are trapped in the dogma of the mechanistic paradigm and the hopelessness of the SDLC. While others will use the OSEA to explore the software universe, others will remain trapped in the mines of reductionism digging for that ultimate technical solution, the “Silver Bullet.”

The End of Scheduling and Software Project Problems

Software project management requires a comprehensive problem solving approach. Currently, project managers lack a clear conception of the problem and solution elements involved in the management of software systems. There is no understanding of the mechanisms and metrics needed to identify and solve problems and take the actions required for adaptation to changing events. As a result, when problems occur project managers can only request more people, time, and money or take actions that will reduce quality or functionality. While the hard quantitative metrics may indicate that there is something a wrong, there is a need for qualitative metrics to understand what it is wrong and why. Without problem solving mechanisms and metrics that improve understanding, projects will continue to become farther behind schedules. Eventually pressure from management will become so great that the project team will implement an immature system.

Turing award winner and author of the “Mythical Man Month”, Fred Brooks stated that software systems are disappointing because they are developed like tangible goods. He argues that the difference is that tangible goods are made of atoms while software systems are made of bits. There is also the change dynamics. Brook argues that software objects are subjected to constant changes during development and after implementation while manufactured things are infrequently changed after manufacture. The problem in trying to manufacture software is that most factories perform visible, repetitious, simple, mechanical tasks. On the other hand, software development is mental work that involves a level of problem solving that is much more uncertain and much less deterministic than factory work. You cannot do time and motion studies on the human brain. Unlike the consistency of factory inputs, requirements as the project’s primary input are of very poor quality and likely to change. Trying to manufacture a product under circumstances where the inputs are often invalid, ambiguous, complex, incomplete, inconsistent and fuzzy is an exercise in futility. System requirements changes and other downstream changes significantly disrupt a deterministic manufacturing process with significant increases in changes and rework that require more effort and time. 

Guru

Open systems project managers need to constantly holistically assess what is out of balance, what teams are functioning effectively, and then make decisions regarding where and how to deploy the time, attention, resources, and energy needed to restore the  out of balance conditions. Project management need advanced, open systems project tracking systems to help them rapidly analyze project situations, identify potential problems, determine the causes, and implement solutions fast enough to keep the project on track. Guru, a prototype of an open system project management approach will be used as an example. Based upon models of similar performance, the prototype will be used to identify out-of balance conditions, analyze the possible situations, rank the potential problems, and identify solutions.

Traditional project management practices control projects through task management. In Guru, it is trend analysis. The purpose of trend analysis is to quickly identify potential problems and make the mid-course corrections before project system objectives are put at risk. Guru will be used to identify out-of balance conditions, analyze the possible situations, rank the potential problems, and identify a range of solutions. The first step is to develop model of expected performance from similar projects, particularly those projects used as the basis for estimating and planning. The idea is to build baseline performance models from clusters of similar systems at user designated checkpoints.

These checkpoints are based upon systems engineering activities that include the systems engineering lifecycle functions. The systems engineering lifecycle functions are requirements and design, acquisition and development, test and validation, deployment, support, training, tech support, operations and maintenance, and software removal. Each activity is decomposed into a series of control points. A user-designated checkpoint is a fraction of an activity 25%, 50%, 75%, or 100% of the software functional elements, particularly the features. Remember software functional consists of systems, subsystems, software functions, capabilities, and features. For example, what should a project look like at 50% of features tested in this release for Team A, Team B, or the entire project?

 

Feature tracking is useful because features are common to all software systems. Features facilitate communication with the business customer and improve the ability to communicate project status in a manner that is understandable to the customer. Contrast the value of feature metrics with LOC and function points. By the end of the systems engineering release planning activity, the number of features planned for release is well known. As a result, the percentage completion and performance ratios are highly accurate because the size factors are known. Percentage completion ratios reflect that features are defined, allocated, designed, coded, integrated, deployed, supported, trained, operated and maintained divided by total features. Performance ratios are expressed in nominator and denominator relationships such as effort/features, defect/features, and changes/features and many more. For example, 1000hours/250features, 2defects/50 features, 10changes/50features. Productivity ratios are the opposite such as 50features/1000 hours.  

Dynamic Variables   

Dynamic variables are the measures allow a project manager to identify when the project is approaching trouble, the probable conditions that are causing the potential problem and a range of solutions used by other projects in the organization to resolve the potential problem. The methodology is to select clusters of similar subsystems, develop baselines measures for the dynamic measure from the clusters, and compare the current project measure to the baseline measures. Baselines of the dynamic variables include the normal range and the abnormal ranges that includes lower than normal and higher than normal. Dynamic variables are the measures of interest to a particular project and could include staff hours, problem reports, percent software functions completed, and staff count as well as features certified, supported, deployed, and satisfied. At the designated control points, Guru graphs the actual values and the baseline values. Abnormal conditions occur when actual values are above or below the normative ranges.

If the current measure is within the boundaries established by the baseline measure, the dynamic variable is good. If the measure is out of balance or trending toward being out of balance (higher than normal or lower than normal), the project manager initiates an investigation. During investigation, an expert function asks a series of questions based upon the possible common causes. Special rules and knowledge gleaned from extensive research into the causes and effects of project deviations can help project managers reach conclusions about the reasons for the out of balance measures. Based upon problem solving processes, lessons learned, profiles, relational clusters, models, and processes, the investigators answers the questions posed by the system for that specific out-of-balance dynamic variable. For example, Staff Hours Greater Than Normal, Staff Hours Lower than Normal, Features tested Higher than Normal, Features Tested Lower than Normal, or Defects Detected High than Normal, Defects Detected Lower than Normal, ectra. 

7.7.1.1 Staff Hours Greater Than Normal Examples

Under these circumstances, analyze the accuracy of the size and effort estimates. If effort estimates are created from any method other than models generated from similar previous projects, estimating is probably the problem.  

Measure the growth and stability of the features designated to be coded in the release. If the number of feature in the release is growing significantly, take actions to stabilize the release. Are features growing or changing? If significant growth has occurred and requirements creep were not factored into the release estimates, re-estimate the release and ensure that the features are required. Are there still undefined requirements or outstanding issues that require resolution? Address the relationships between the known and unknown features. If possible, put the unstable features on hold and address the known features. Schedule the implementation of the remaining features once the requirements are known.

If the size growth is insignificant then the problem is productivity. Calculate development team productivity by dividing the number of features completed by number of staff hours required to complete the work units. Compare the results to the productivity of the projects sued for estimating. Compare the original staffing profile to actual staff profiles. Match original job category and skill requirements with those of the project team. If the staffing profile is the problem, re-estimate the project based upon the current staffing profile. Evaluate supervisory skills, work environment, and team effectiveness. Perform a root-cause analysis. Analyze team domain, application, language, and tool experience. If the problem is development team experience adjust the project accordingly. Analyze the management plans, poor planning could result in the decrease in productivity. Evaluate the quality of the plans and determine whether the team follows the plans. Rate the quality of the various plans. Finally, consider that the productivity assumptions were too ambitious.

7.7.1.2 Staff Hours Lower Than Normal Examples

 

If staff hours are below normal and schedules are not being met, then the project does not have enough people and is staffing up too slowly. Compare the current staff profile to the original. Make adjustments that reflect the differences in staffing assumptions and staffing actuality. 

If staff hours are below normal and the team continues to meet schedules, the problem class context is lower than expected. In other words, the problem is simpler than expected, the system is easier than expected, the system is smaller than expected, or the teams are better than expected.  

However, it could also mean that the development team lacks complete understanding of the problem and requirements or they are maintaining schedules at the expense of quality. Determine whether team and user have clear understanding of the problem and the requirements. Ensure that the team is not trying to maintain schedules at the expense of quality. They might be neglecting defect prevention, detection, and correction activities. Make sure that team is effectively performing inspections, code reading, walk-through, testing and other feedback control methods.   

Joint Optimization  

Joint optimization directs time, effort, and resources to the most important, common, high usage, high priority software functional elements (systems, subsystems, software functions, capabilities, and features). System effectiveness must come first but the marginal and probably unused, lowest priority, software functional elements must compete with the project’s resources, budgets, and scheduling constraints. Under this evolutionary development approach, user certified software functional elements are delivered in five stages of software complexity: the basic or structural, the mandatory, the mature, the elegant, and the sophisticated. With joint optimization, the definition of failure shifts from realizing a system that is very far over budget and behind schedule, of very poor quality, or needs extensive rework to projects that are mature, on time and within budget, and of user certified quality sans the often unused and unneeded bells and whistles features.

Joint optimization is possible because not all requirements are equal. A study by the Standish Group of 100 companies found that 45% of application features are never used, 19 percent rarely used, 16 percent sometimes used, 13 percent often used, and 7 percent always used. Joint optimization prioritizes the major the software functional elements so that the bulk of the project resources go to the most highly prized functional elements. The more marginal and lesser needed functional elements must compete with project efficiency in terms of schedules and costs. As a result, the scarce resources no longer automatically flow to the development of unused or unneeded functionality that exponentially adds to project costs, schedules, and complexity. For example, systems architectures designed around unused or rarely used functionality also increases the maintenance and operational costs of systems.

Without the marginal requirements, highly robust and quality systems could be implemented in less than 1/2 the time of monolithic projects. Concurrent software engineering requires that all the system engineering lifecycle functions be released concurrently. These top-tier software functional elements will receive the best quality and the best system support in terms of deployment, training, tech support, operations and maintenance. This will improve systems efficiency. No longer will software be considered too difficult and expensive to setup, tech support, document, deploy, train, operate and maintain because the project ran out of time and money developing excessive functionality

Gone are the massive wastes from excessive functionality that go unused, sometime as much as 45% of the feature set. Gone are the projects that seem to go on forever amassing tremendous schedule and budget overruns because there were no distinctions between software functional elements that were important and those that were not. Gone are the additional resources from unnecessary increase in system size and the accompanying exponential growth in project complexity, difficulty, time, costs, and effort. Gone are the unused, rejected, cancelled, or low value system that did not achieve the desired results because each release will be certified before another is formally started. This way the horror of implementing a system that does not work is avoided. Gone are the project cancellations with nothing to show for the expended time, costs, and effort. Unused or rejected systems are in the past because each release will be certified before another is formally started. Gone are the costs of operational software systems are inflexible, difficult to use, expensive to support, hard to deploy, and very costly to maintain because the software systems and support systems are released and delivered together.

Finally, gone is the schedule and budget waste caused by complex and difficult software designs. Organic software objects are many times less complex and difficult to deliver than machine objects because machine objects group similar requirements from different, dissimilar, and distinct sources into centralized designs that quickly turn into spaghetti code. Centralized designs serve many too many constituents are difficult to visualize, design, code, and test as well as being costly to maintain and error prone. The lack of visibility, conformity, and changeability increase problems and extend budgets and schedules.