What is Batch Processing ?

Batch processing is execution of a series of programs ("jobs") on a computer without manual intervention.

Batch jobs are set up so they can be run to completion without manual intervention, so all input data is preselected through scripts orcommand-line parameters. This is in contrast to "online" or interactive programs which prompt the user for such input. A program takes a set of data files as input, process the data, and produces a set of output data files. This operating environment is termed as "batch processing" because the input data are collected into batches on files and are processed in batches by the program.

Benefits of Batch Processing:

• It allows sharing of computer resources among many users and programs
• It shifts the time of job processing to when the computing resources are less busy
• It avoids idling the computing resources with minute-by-minute manual intervention and supervision
• By keeping high overall rate of utilization, it better amortizes the cost of a computer, especially an expensive one.

Common Batch Processing Usage:

Data processing:

A typical batch processing procedure is End of day-reporting (EOD), especially on mainframes. Historically systems were designed to have a batch window where online subsystems were turned off and system capacity was used to run jobs common to all data (accounts, users or customers) on a system. In a bank, for example, EOD jobs include interest calculation, generation of reports and data sets to other systems, print (statements) and payment processing.

Printing:

A popular computerized batch processing procedure is printing. This normally involves the operator selecting the documents they need printed and indicating to the batch printing software when, where they should be output and priority of the print job. Then the job is sent to the print queue from where printing daemon sends them to the printer.

Databases:

Batch processing is also used for efficient bulk database updates and automated transaction processing, as contrasted to interactive online transaction processing (OLTP) applications.

Images:

Batch processing is often used to perform various operations with digital images. There exist computer programs that let one resize, convert, watermark, or otherwise edit image files.

Converting:

Batch processing is also used for converting a number of computer files from one format to another. This is to make files portable and versatile especially for proprietary and legacy files where viewers are not easy to come by.

Difference between Supercomputers and Mainframes

Supercomputers are very different from mainframe computers. A supercomputer is a computer that is at the frontline of current processing capacity, particularly speed of calculation. Supercomputer can be based on any type of computer design (e.g., FPGA, vector processing,microprocessor, and GPGPU based computer).Supercomputers are used for scientific and engineering problems which are limited by processing speed and memory size, while mainframes are used for problems which are limited by input/output and reliability and for handling multiple business transactions concurrently. The differences are as follows:

• Mainframes are measured in integer operations per second or MIPS; whereas, Supercomputers are measured in floating point operations per second or FLOPS. Example of integer operation is moving data around in memory or adding integers. Example of floating point operation is the calculation of mathematical equations in real numbers. In terms of computational ability, Supercomputers are more powerful. Mainframes are built to be reliable for transaction processing as it is commonly understood in the business world: a commercial exchange of goods, services, or money. A typical transaction would include the updating to a database system for such things as inventory control (goods), airline reservations (services), or banking (money). A transaction could refer to a set of operations including disk read/writes, operating system calls, or some form of data transfer from one subsystem to another.
• Both types of systems offer parallel processing, although this has not always been the case. Parallel processing (i.e., multiple CPUs executing instructions simultaneously) was used in supercomputers (e.g., the Cray-1) for decades before this feature appeared in mainframes. Supercomputers typically expose parallel processing to the programmer in complex manners, while mainframes typically use it to run multiple tasks. One result of this difference is that adding processors to a mainframe often speeds up the entire workload transparently, with important exceptions of single batch jobs and CICS regions that don't run faster.
• Supercomputers are often built to run complex algorithm in wide application area such as in industry, academic, research, government or military (e.g., nuclear simulation and modeling, life science, medicine, telecommunication, weather forecasting), while mainframes run simple algorithm to handle variety of tasks (e.g. database query, warehousing, inventory, operating system). Supercomputer uses both off-design technology (e.g.: microprocessor based cluster computer) or advanced technology (e.g., vector processing, NUMA, liquid-cooling); whereas mainframes typically form part of a manufacturer's standard model lineup.
• Mainframes tend to have numerous ancillary service processors assisting their main central processors (for cryptographic support, I/O handling, monitoring, memory handling, etc.) so that the actual "processor count" is much higher than would otherwise be obvious. Supercomputer design tends not to include as many service processors since they don't appreciably add to raw number-crunching power.
• Mainframes are exceptionally adept at batch processing, such as billing, owing to their heritage, decades of increasing customer expectations for batch improvements. Supercomputers are very expensive and not cost-effective just to perform batch or transaction processing.

General History of Mainframes

Several manufacturers produced mainframe computers from the late 1950s through the 1970s. The group of manufacturers was first known as "IBM and the Seven Dwarfs": IBM, Burroughs, UNIVAC, NCR, Control Data, Honeywell, General Electric and RCA. Later, shrinking, it was referred to as IBM and the BUNCH. IBM's dominance grew out of their 700/7000 series and, later, the development of the 360 series mainframes.

Notable manufacturers outside the USA were Siemens and Telefunken in Germany, ICL in the United Kingdom, and Fujitsu,Hitachi, Oki, and NEC in Japan. The Soviet Union and Warsaw Pact countries manufactured close copies of IBM mainframes during the Cold War; the Strela is an example of an independently designed Soviet computer.

Shrinking demand and tough competition caused a shakeout in the market in the early 1980s — RCA sold out to UNIVAC and GE also left; Honeywell was bought out by Bull; UNIVAC became a division of Sperry, which later merged with Burroughs to form Unisys Corporation in 1986. In 1991, AT&T briefly owned NCR. During the same period, companies found that servers based on microcomputer designs could be deployed at a fraction of the acquisition price and offer local users much greater control over their own systems given the IT policies and practices at that time. Terminals used for interacting with mainframe systems were gradually replaced by personal computers. Consequently, demand plummeted and new mainframe installations were restricted mainly to financial services and government. In the early 1990s, there was a rough consensus among industry analysts that the mainframe was a dying market as mainframe platforms were increasingly replaced by personal computer networks. Infoworld's Stuart Alsop famously predicted that the last mainframe would be unplugged in 1996.

That trend started to turn around in the late 1990s as corporations found new uses for their existing mainframes and as the price of data networking collapsed in most parts of the world, encouraging trends toward more centralized computing. The growth of e-business also dramatically increased the number of back-end transactions processed by mainframe software as well as the size and throughput of databases. Batch processing, such as billing, became even more important (and larger) with the growth of e-business, and mainframes are particularly adept at large scale batch computing. Another factor currently increasing mainframe use is the development of the Linux operating system, which arrived on IBM mainframe systems in 1999 and is typically run in scores or hundreds virtual machines on a single mainframe. Linux allows users to take advantage of open source software combined with mainframe hardware RAS. Rapid expansion and development in emerging markets, particularly China, is also spurring major mainframe investments to solve exceptionally difficult computing problems, e.g. providing unified, extremely high volume online transaction processing databases for 1 billion consumers across multiple industries (banking, insurance, credit reporting, government services, etc.) In late 2000 IBM introduced 64-bit z/Architecture, acquired numerous software companies such as Cognos and introduced those software products to the mainframe.

Characteristics of Mainframes

Nearly all mainframes have the ability to run (or host) multiple operating systems, and thereby operate not as a single computer but as a number of virtual machines. In this role, a single mainframe can replace dozens or even hundreds of smaller servers. While mainframes pioneered this capability, virtualization is now available on most families of computer systems, though not always to the same degree or level of sophistication.

Mainframes can add or hot swap system capacity non disruptively and granularly, to a level of sophistication usually not found on most servers. Modern mainframes, notably the IBM zSeries, System z9 and System z10 servers, offer two levels of virtualization: logical partitions (LPARs, via the PR/SM facility) and virtual machines (via the z/VM operating system). Some IBM mainframe customers run no more than two machines : one in their primary data center, and one in their backup data center—fully active, partially active, or on standby—in case there is a catastrophe affecting the first building. Test, development, training, and production workload for applications and databases can run on a single machine, except for extremely large demands where the capacity of one machine might be limiting. Such a two-mainframe installation can support continuous business service, avoiding both planned and unplanned outages. In practice most customers use multiple mainframes linked by Parallel Sysplex and shared DASD.

Mainframes are designed to handle very high volume input and output (I/O) and emphasize throughput computing. Since the mid-1960s, mainframe designs have included several subsidiary computers (called channels or peripheral processors) which manage the I/O devices, leaving the CPU free to deal only with high-speed memory. It is common in mainframe shops to deal with massive databases and files. Giga-record or tera-record files are not unusual.Compared to a typical PC, mainframes commonly have hundreds to thousands of times as muchdata storage online, and can access it much faster.Other server families also offload I/O processing and emphasize throughput computing.

Mainframe return on investment (ROI), like any other computing platform, is dependent on its ability to scale, support mixed workloads, reduce labor costs, deliver uninterrupted service for critical business applications, and several other risk-adjusted cost factors.

Mainframes also have execution integrity characteristics for fault tolerant computing. For example, z900, z990, System z9, and System z10 servers effectively execute result-oriented instructions twice, compare results, arbitrate between any differences (through instruction retry and failure isolation), then shift workloads "in flight" to functioning processors, including spares, without any impact to operating systems, applications, or users. This hardware-level feature, also found in HP's NonStop systems, is known as lock-stepping, because both processors take their "steps" (i.e. instructions) together. Not all applications absolutely need the assured integrity that these systems provide, but many do, such as financial transaction processing.

Description of Mainframe computers

Modern mainframe computers have abilities not so much defined by their single task computational speed (usually defined as MIPS — Millions of Instructions Per Second) as by their redundant internal engineering and resulting high reliability and security, extensive input-output facilities, strict backward compatibility with older software, and high utilization rates to support massive throughput. These machines often run for years without interruption, with repairs and hardware upgrades taking place during normal operation.

Software upgrades are only non-disruptive when Parallel Sysplex is in place, with true workload sharing, so one system can take over another's application, while it is being refreshed. More recently, there are several IBM mainframe installations that have delivered over a decade of continuous business service as of 2007, with hardware upgrades not interrupting service. Mainframes are defined by high availability, one of the main reasons for their longevity, because they are typically used in applications where downtime would be costly or catastrophic. The term Reliability, Availability and Serviceability (RAS) is a defining characteristic of mainframe computers. Proper planning (and implementation) is required to exploit these features.

In the 1960s, most mainframes had no interactive interface. They accepted sets of punch cards, paper tape, and/or magnetic tape and operated solely in batch mode to support back office functions, such as customer billing. Teletype devices were also common, at least for system operators. By the early 1970s, many mainframes acquired interactive user interfaces and operated as timesharing computers, supporting hundreds or thousands of users simultaneously along with batch processing. Users gained access through specialized terminals or, later, frompersonal computers equipped with terminal emulation software. Many mainframes supported graphical terminals (and terminal emulation) by the 1980s (if not earlier). Nowadays most mainframes have partially or entirely phased out classic terminal access for end-users in favor of Web user interfaces. Developers and operational staff typically continue to use terminals or terminal emulators.

Historically, mainframes acquired their name in part because of their substantial size, and because of requirements for specialized heating, ventilation, and air conditioning (HVAC), and electrical power. Those requirements ended by the mid-1990s with CMOS mainframe designs replacing the older bipolar technology. In a major reversal, IBM now touts its newer mainframes' ability to reduce data center energy costs for power and cooling, and the reduced physical space requirements compared to server farms.

Mainframe computer

Mainframes (often colloquially referred to as Big Iron) are computers used mainly by large organizations for critical applications, typically bulk data processing such ascensus, industry and consumer statistics, enterprise resource planning, and financial transaction processing.

The term originally referred to the large cabinets that housed the central processing unit and main memory of early computers. Later the term was used to distinguish high-end commercial machines from less powerful units.

In 2009, the term refers to computers compatible with the IBM System/360line, first introduced in 1965. (IBM System z10 is the latest incarnation.) Otherwise, large systems that are not based on the System/360 but are used for similar tasks are usually referred to as servers. However, "server" and "mainframe" are not synonymous (see client-server).

Some non-System/360-compatible systems derived from or compatible with older (pre-Web) server technology may also be considered mainframes. These include the Burroughs large systems, the UNIVAC 1100/2200 series systems, and the pre-System/360 IBM 700/7000 series. Most large-scale computer system architectures were firmly established in the 1960s and most large computers were based on architecture established during that era up until the advent of Web servers in the 1990s. (Interestingly, the first Web server running anywhere outside Switzerland ran on an IBM mainframe at Stanford University as early as 1990. See History of the World Wide Web for details.)

There were several minicomputer operating systems and architectures that arose in the 1970s and 1980s, but minicomputers are generally not considered mainframes. (UNIX arose as a minicomputer operating system; Unix has scaled up over the years to acquire some mainframe characteristics.)

Many defining characteristics of "mainframe" were established in the 1960s, but those characteristics continue to expand and evolve to the present day.

HI

Welcome to this Blog..

I'll post periodically about Mainframes & it's programming..