LifeFormulae Blog

Another Study on What Women Really Want

Pam — Tue, 24 Aug 2010 20:53:59 +0000

First, we had the guy from Harvard try to explain that women aren’t interested in science because there is an intrinsic aptitude for things scientific based on gender. Guess which gender is deemed as more scientific?

Now, we have a new observation brought to us by Wray Herbert (http://www.huffingtonpost.com/wray-herbert/women-science_b_652858.html).

According to Miami University psychological scientist Amanda Dickman, there is a new explanation citing a difference in worthiness or values rather than ability. It seems, according to the new theory, that women reject science, engineering, and math because they view the these fields as too ego and power driven for their tastes.

The unambiguous results for the study found that young women did see science and engineering careers as isolated and individualistic–and what’s more, as obstacles to finding meaning in their lives.

The article goes on to state that it seems to be a perception thing. I would agree that it could very well be the perception thing, but there I think there is a little more to it than that.

A Little Background

My higher education endeavors began with a trip down the road that would merit approval from the study group quoted above. I got an undergraduate degree in Social and Behavioral Sciences and was just a few hours away from a graduate degree when I discovered I was bored to death. Something was missing. There was no challenge.

I tried the MBA path. Nothing doing.

I had taken an intro to computers course as part of my undergraduate course work and a ton of statistics courses but neither appealed. It wasn’t until I ran into my first “micro-computer” (as they were then known), that I realized this little machine was really going to change things. I even got a Heath kit catalog, ordered the H-89 kit, and put it together.

The closest decree to a computer science degree my university offered was a degree in mathematical sciences. I signed up for that.

Believe me, it wasn’t easy. I had already gotten the required courses out of the way, so for three semesters every class I had was either math or computer science. But it was interesting and definitely challenging.

Group Members

The isolated and individualistic scientist, engineer, computer scientist as cited by the study does not exist in the real world.

My first post graduation gig was at the Health Services Division of a major aerospace company as a compiler developer. I was part of the Systems Enhancements and Extensions Group. From there, I transferred to the aircraft company in that same corporation. I was part of the Flight Test Research and Development Group. I went to another aircraft company and the Instrumentation Group. And so on. You were always a member of a group. A group that together designed, developed, and produced things – computer software, digital data acquisition systems, aircraft manufacturing scheduling systems, etc.

When I moved over to biotechnology, it was the same – you were a member of a group. A lab group, a bioinformatics group developing LIMS systems, sequence analysis and imaging recognition software, and so on.

However, I did find that scientists more that engineers were more power/ego driven. I think this is because of funding issues. Although both areas receive the majority of their funds from the government, the basis of the awards is different.

The individual scientist, as P.I., applies for the grant, writes the proposal and receives the funding – almost a personal assessment of that scientist’s capabilities. Furthermore, I feel that the letters - “PhD”, carries a lot of baggage.

For most engineers, the company applies for the grant, writes the proposal (after the engineers have okayed the design), and receives the funding. The engineer is associated with the program for which that proposal was submitted. The engineer isn’t as personally involved.

What I’ve Encountered

In the military industrial complex I encountered bored ex-military who used weekly status reports to declare war on some other part of the division . These attacks were mostly diversions and never amounted to much. These could be construed as power plays, but I list them as “play” period.

Believe me, there were some good ones – stopping just short of an exchange of blows. It’s also amazing how far echoes carry in an aircraft hanger.

The following examples are situations I encountered along the way. They are mostly examples of misdirected intentions, but a few border on outright criminality.

There were approximately 8 databases that all held the same information but for 8 different divisions. The electronics parts – transducers, potentiometers, strain gauges, resistors etc, in each of the databases were exactly the same. However, the nomenclature varied by division. We tried to standardize on one database system with one naming standard, but ran straight into a brick wall. Not one division was willing to cede to another. It was only after word came down from on high that additional funding would not be forthcoming, that everybody finally sat down to talk.

Insane Budgeting Exercises

One division needed to get a new system but was offered an old barely breathing system with exorbitant maintenance costs. The division was instructed to budget for and use the old system for the current fiscal year. For the he next budget cycle, the department was to state that a new system (the one originally requested) would save X amount of dollars over last year’s budget. The new system was then be given the green light.

A director was undercutting his yearly budget to emphasize cost savings. Consequently, his budget was always cut to that amount for the next year. It was pointed out that he should over run this year’s budget by the amount he wanted for next year. Then he would (and did) get the additional funding.

A Simple Name Change can Work Wonders

it was ascertained that for less that the amount the department was paying IT for storage of design data, a new system, software, and personnel could be purchased and hired. Department was notified that requesting a “computer system” would not meet with budgeting approval Only after the system was termed a “data multiplexer” to be administered by “data design personnel” was department able to proceed with system purchase.

One Size Does Not Fit All

IT sends down list of “acceptable” software. So-called software was specifically IT oriented and would not work in an engineering environment. Division engineers take up collection and purchase needed software themselves.

Almost Criminal

Vast amounts of money, time, manpower were spent developing a manufacturing scheduling system for aircraft manufacture. System rated manufacturing personnel in terms of ability. System was deemed a major success – avoiding bottlenecks, completion times, etc. System was never deployed due to union demands that manufacturing personnel could not be rated in terms of ability.

Decode system purchased for data acquisition decode and analysis ($150K) was purchased without installed hard drive for data storage ($15K). It was determined system could use in-house data farm to store data. Decode system required confirmation that contiguous data storage space was available to go ahead and store data.

Transfer mechanism did not provide this info, so decode system would not store data on data farm. Contractor told department officials that the system software on the decode system and in-house data farm were incompatible. Contractor sold department customized software for $750K to replace decode system.

A Meaningful Life

I’ve never considered my career in engineering and biotechnology as isolated and individualistic. Sure, you have individual work, but it is as part of a team.

As far as letting the ego and power driven become obstacles, I have to admit that my behavioral sciences background provided one of the most important career tools I have yet to encounter. My “Advanced Abnormal Psychology” course taught me how to observe and analyze people.

To find meaning in one’s life entails one heck of a lot more than a career. Perhaps by observing and analyzing one’s misconceptions about one area will enhance our conceptions of life in general.

What is Old Is New Again - The Cloud

Pam — Fri, 21 May 2010 20:44:05 +0000

Cloud computing is the current IT rage, said to cure all information management skills.

Cloud computing is just a new name for timeshare, a system in which various entities shared a centralized computing facility. A giant piece or two of big iron and floors of tape decks provided information processing and storage capabilities for a price.

The user was connected to the mainframe by a dumb terminal and later on by PC’s. The advantage (said the sales jargon), was that the user didn’t need to buy any additional hardware, worry about software upgrades or data backup and recovery. They would only pay for the time and space their processes required. Resources would be pooled and connected by a high speed network and could be accessed as demanded. The user wouldn’t really know what computing resources were in use, they just got results. Everything depended on the network communications between the use and centralized computing source.

What is New

Cloud computing is more powerful today because the communications network is the Internet. Some Cloud platforms also offer Web access to the tools – programming language, database, web utilities needed to create the cloud application.

The most important aspect I believer the Cloud offers is instant elasticity. A process can be upgraded almost instantaneously to use more nodes and obtain more computing power.

There are quite a few blog entries out there concerning the “elastic” cloud. For thoughts on “spin up” and “spin down” elasticity see http://timothyfitz.wordpress.com/2009/02/14/cloud-elasticity/. For thoughts on “how elasticity could make you go broke, or On-demand IT overspending” see http://blogs.gartner.com/daryl_plummer/2009/03/11/cloud-elasticity-could-make-you-go-broke/.

And finally, an article that spawned the “elasticity is a myth” connotation or “over-subscriptionand over-capacity are two different things, see – http://www.rationalsurvivability.com/blog/?p=1672&cpage=1#comment-35881.

A good article that covers elasticity, hypervisors, and cloud security in general is located at http://queue.acm.org/detail.cfm?id=1794516. The queue.acm.org site is maintained by the Association for Computing Machinery. There are lots of articles on all sorts of computing topics including, “Why Cloud Computing Will Never Be Free” (http://queue.acm.org/detail.cfm?id=1772130).

The Clouds

The most notable Clouds are Amazon’s Elastic Cloud, Google’s App Engine, and Microsoft’s Azure.

The three Cloud delivery models include:

- Software as a service (SaaS), applications running on a cloud are accessed via a web browser
- Platform as a service (PaaS), cloud-developed user applications such as databases
- Infrastructure as a service (IaaS), provides computing resources to users on an as-needed basis

Pros and Cons

There are pros and cons for Cloud Computing. Microsoft’s Bill Ballmer is a proponent of Cloud computing.

In a recent email (http://blog.seattlepi.com/microsoft/archives/196793.asp) to Microsoft’s employees, Ballmer make the following case for Cloud Computing. He advises his employees to watch a video (http://www.microsoft.com/presspass/presskits/cloud/videogallery.aspx) in which he makes the following points.

In my speech, I outlined the five dimensions that define the way people use and realize value in the cloud:

The cloud creates opportunities and responsibilities
The cloud learns and helps you learn, decide and take action
The cloud enhances your social and professional interactions
The cloud wants smarter devices
The cloud drives server advances that drive the cloud

Some very notable people are anti-cloud.

Richard Stallman, GNU software founder, said in recent interview for the London Guardian (http://www.guardian.co.uk/technology/2008/sep/29/cloud.computing.richard.stallman) that Cloud computing is a trap.

The Web-based programs like Google’s Gmail will force people to buy into locked, proprietary systems that will cost more and more over time, according to the free software campaigner.

‘It’s stupidity. It’s worse than stupidity: it’s a marketing hype campaign,’ he told The Guardian. ‘Somebody is saying this is inevitable — and whenever you hear somebody saying that, it’s very likely to be a set of businesses campaigning to make it true.’”

Aside from all that, what should a potential user be wary of in the Cloud? I’ll try to answer that below.

Security in the Cloud

Security in the cloud is a major concern. Hackers are salivating because everything – applications, data, are all in the same place.

How do you know the node your process is accessing is real or virtual? The Hypervisor (in Linux, a special version of the kernel) owns the hardware and spawns virtual nodes. If the Hypervisor is hacked, the hacker owns all the nodes created by it. http://www.linux-kvm. org has further explanations and discussions of virtual node creators/creations.

Data separation is a big concern. Could your data become contaminated by data in other environments in the cloud.? What access restrictions are in place to protect sensitive data?

Can a user in another cloud environment inadvertently or intentionally get access to your data?

Data interoperability is another question mark. A company cannot transfer data from a public cloud provider, such as Amazon, Microsoft, or Google, put it in a private IaasP that a private cloud provider develops for a company, and then copy that data from its private cloud to another cloud provider, public or private. This is difficult because there are no standards for operating in this hybrid environment.

Data Ownership

Who is the custodian and who controls data if your company uses cloud providers, public and private?

Ownership concerns have no been resolved by the cloud computing industry. At the same time, the industry has no idea when a standard will emerge to handle information exchanges.

W3C – http://www.w3.org/, is sponsoring workshops and publishing proposals concerning standards for the Cloud. You can subscribe to their weekly newsletter and stay up on all sorts of web-based technologies.

Also, the Distributed Management Task Force Inc.(http://www.dmtf.org/home) is a consortium ofof IT companies focusing on, “Developing management standards & promoting
interoperability for enterprise & Internet environments”.

The DMTF Open Cloud Standards Incubator was launched to address management interoperability for Cloud Systems (http://www.dmtf.org/about/cloud-incubator). The DMTF leadership board currently includes AMD, CA Technologies, Cisco, Citrix Systems, EMC, Fujitsu, HP, Hitachi, IBM, Intel, Microsoft, Novell, Rack Space, RedHat, Savvis, Sun Guard, Sun Microsystems, and VMWare.

Working with the Cloud

Working with the Cloud can be intimidating. One suggestion is to build a private cloud in-house before moving on to the public cloud.

However, even that has its difficulties. Not to worry, there are several tools available to ease the transition.

There is a Cloud programming language – Bloom, developed at UC Berkeley by Dr. Joseph Hellerstein. HPC In The Cloud has published an interview with Dr. Hellerstein at http://www.hpcinthecloud.com/features/Clouds-New-Language-Set-to-Bloom-92130384.html?viewAll=y

Bloom is based on Hadoop (http://hadoop.apache.org) which is open source software for High Performace Computing (HPC) from Apache..

For ease of inter connectivity, Apache has released Apache libcloud, a standard client library written in python for many popular cloud providers – http://incubator.apache.org/libcloud/index.html. But libcloud doesn’t cover data standards, just connectivity.

MIT StarCluster– http://web.mit.edu/stardev/cluster , is an open source utility for creating and managing general purpose computing clusters hosted on Amazon’s Elastic Compute Cloud (EC2). StarCluster minimizes the administrative overhead associated with obtaining, configuring, and managing a traditional computing cluster used in research labs or for general distributed computing applications.

All that’s needed to get started with your own personal computing cluster on EC2 is an Amazon AWS account and StarCluster.

HPC presents use cases as a means to understanding cloud computing. http://www.hpcinthecloud.com/features/25-Sources-for-In-Depth-HPC-Cloud-Use-Cases-93886489.html.

BCM Bioinformatics has a new methodology article – Cloud Computing for Comparative Genomics that includes a cost analysis of using the cloud. Download the .pdf at http://www.biomedcentral.com/1471-2105/11/259/abstract.

I hope this will get you started. Once again, a big thanks to Bill for his assistance.

Effective Bioinformatics Programming - Part 5

Pam — Tue, 06 Apr 2010 17:55:37 +0000

First, a little irony. In the late ’90’s I interviewed with BMC software in Houston. At that time, BMC was a supporter of big iron, providing report facilities, etc.

When asked what software I currently used, I replied with “GNU software”. The interviewer asked, “What is GNU? I’ve never heard of it.”

I explained that it was free software that you could download from the web, etc. But they weren’t really interested.

Anyway, eWEEK.com had a feature this week - ‘MindTouch Names 20 Most Powerful Open-Source Voices of 2010. The first name mentioned was William Hurley. The chief architect of Open Source strategy at BMC. (http://www.eweek.com/c/a/IT-Management/OSBC-Names-20-Most-Powerful-Open-Source-Voices-of-2010-758420/?kc=EWKNLEDP03232010A).

I guess they’re interested now.

Data Standards

There are any number of sequence data formats. This link at EBI – http://www.ebi.ac.uk/2can/tutorials/formats.html describes several.

What is really astounding is that most of these formats have remained to same over the years. The tab-delimited and CSV (comma separated values) format is as prolific as ever, as is the GenBank report.

And equally astonishing is the fact that manipulating the data (e.g. parsing GenBank reports) is still the same.

True, the Bio libraries such as BioPerl, BioJava, BioRuby, now provide modules that make this easier, (if you can install them) but it is still the same old download and parse.

There are also several groups trying to standardize sequence data. The SO (Sequence Ontology) group (http://www.sequenceontology.org) is trying to do for sequence annotations what GO (Gene Ontology - http://www.geneontology.org) did for genes and gene product attributes.

MIGS (Minimum Information About A Genome Sequence spec at http://nora.nerc.ac.uk/5548/) is following the course of the MAGE MIAME Standard (Minimum Information About a Microarray Experiment at http://www.mged.org/Workgroups/MIAME/miame.html). Good luck with that, as many scientists have openly voiced objections to that standard.

XML and the Web

XML (eXtensible Markup Language) and WSDL (Web Services Description Language) are one method of easing the interchange of data. Links at – http://en.wikipedia.org/wiki/XML and http://en.wikipedia.org/wiki/Web_Services_Description_Language.

There are a number of drawbacks to this setup.

Not all of the sequence data is available in XML or well-formed XML.

Some XML, such as NCBI XML, needs further interpretation. For example, the sequence feature (annotation) locations must be “translated” for further use.

XSLT has performance issues, and is size-delimited. We tried processing LARTS converted NCBI ASN.1 GenBank XML data to XSLT and found there were definite size limitations.

Using WSDL means exposing yourself to the world via the web.

Javascript has too many security questions to consider seriously.

Software Development

Software development takes time and the right people. True, there is a lot of open source software out there, but I’ve mentioned the perils of that method in a previous blog.

A scientist with a grant to produce results dependent on computer analysis is only going to write code that is good enough to create code (or find someone (read post-doc) who can create that code very cheaply) that will back up those findings.

Has the code been extensively tested? Are the results produced by the code valid? Can the code be used by future projects? Is the software portable? Is it robust? Can it be ported to different hardware environments?

There is a great article – “Are we taking supercomputing code seriously?” at (http://www.zdnet.co.uk/news/it-strategy/2010/01/28/are-we-taking-supercomputing-code-seriously-40004192/). This article, in turn, has links to other articles on methods and algorithms, and error behavior, for example. This one on scientific software considers how multi-processing has influenced algorithm development and the problem of different multi-processors co-existing on the same machine (http://www.scientific-computing.com/features/feature.php?feature_id=262).

He states that in the rush to do science, scientists fail to spot software for what it is: the analogue of the experimental instrument. Therefore the software must be treated with the same respect that a physical experiment would.

When I started my career, I worked on a system that was a totally integrated database system for hospitals. It was one of those systems that was so very ahead of its time (mid-80’s), that a corporation bought the product and squashed it.

Anyway, our Systems and Extensions group supported the 6 compilers that comprised the system software that made the system function. The tailoring group wrote the code that created the screens that drove the system.

At the inception of the system, a decision was to be made over the make up of the tailoring group: should they be programmers that would be taught medical jargon, terms, etc; or should they be medical personnel – doctors, nurses, techs, that would be taught programming?

The decision was to go with medical personnel, as it was surmised they would understand hospitals better.

At the same time, a decision to limit the number of screens a hospital could request (called tailoring) to 500 was discussed. The decision was to let the hospital have however many screens it wanted.

The tailoring group got their training and set in to programming. After a period of time, it was realized that the group had, in essence, created one bad program and copied it thousands of times.

It was so bad, we did two things. We created a program profiler that produced a performance summary of the programming aspects of that program. (We were immediately asked to remove it by the tailoring group, as it was too confusing.) Two, we created an automated programming module that would create the code from the display widgets on the screen designed by the tailoring group.

This approach was helping, but people were abandoning ship as talk of an acquisition was surfacing. Our junior programmer went from new-hire to senior team member in 30 days.

I think we would have done a lot better with programmers learning medical terms.

As for the hospital screen limit, we had hospitals with 10,000 individual screens. We should have stuck with 500.

One last thing. When looking at any piece of scientific programming, please realize that in the Authors accreditation usually starts with the PI. The people who did the actual work are generally listed at the end of the line. The PI may have had the idea, but likely as not could not code it.

Effective Bioinformatics Programming Part 4

Pam — Wed, 10 Mar 2010 19:46:20 +0000

All Things HPC

Traditionally, High Performance Computing (HPC) means using high-end hardware like super computers to perform complex computational tasks.

A new definition of HPC (“High Productivity Computing”) means the entire processing and data handling infrastructure. This includes software tools, platforms (computer hardware and operation systems), and data management software.

Parallel or Multicore Processing

I think just about everybody has performed some sort of parallel programming. Starting two processes at once on the same machine is parallelism. If the program runs by itself and doesn’t need input from another program or product output for another program to use, it’s loosely coupled. It’s tightly coupled if one program feeds another.

PC architecture today supports multicore processors. A two-core CPU is, in essence, two CPUs on the same chip. These cores may share memory cache (tightly coupled) or not (loosely coupled). They may implement a method of message passing – intercore communication.

Cilk is a language for multi-threaded parallel processing based on ANSI C. MIT was the initial developer of the Cilk technology. The link to their page is at – http://supertech.csail.mit.edu/cilk/.

MIT had licensed Cilk to Cilk Arts, Inc. Cilk Arts added support for C++, parallel loops, and interoperability with serial interfaces. The product has since been acquired by Intel and will be incorporated into the Intel C++ compiler. The Intel page is at http://software.intel.com/en-us/articles/intel-cilk/.

Cilk++ makes multicore processing easy. CILK++ uses keywords to adapt existing C++ code to multicore processing. (You will need a multi-core processor).

Cilk++ is currently is a technical preview state. This means they want you to use it and give them feedback. Download the Intel CILK++ SDK at http://software.intel.com/en-us/articles/download-intel-cilk-sdk/. You will need to sign a license agreement.

The page also presents download links for 32-bit and 64-bit Linux Cilk++. (You will need an Intel processor for the Linux apps.)

There is an e-book on Multi-Processor programming available from Intel. The link is - http://software.intel.com/en-us/articles/e-book-on-multicore-programming/.

The book contains a lot of information on multicore programming, parallelism, scheduling theory, shared memory hardware, concurrency platforms, race conditions, divide and conquer recurrences, and others.

Grid computing is distributed, large scale, cluster computing. Two of the most famous grid projects are SETI@home and Folding@home (http://folding.stanford.edu).

SETI (the search for extra-terrestrial Intelligence) at home uses internet connected computers hosted by Space Sciences Laboratory at the UC, Berkeley. Folding at home focuses on how proteins (biology’s workhorses) fold or assemble themselves to carry out important functions.

Other lesser known grids are einstein@home (http://www.einsteinathome.org - “Grab a wave from Space”) processing data from gravational wave detectors, and MilkyWay@home (http://milkyway.cs.rpi.edu/milkyway) creating a highly accurate 3-D model of the Milky Way Galaxy.

Communication

The clusters mentioned above use the internet to exchange messages. If fast messaging is not required, plain old ethernet should be sufficient for your messaging needs, The problem with ethernet is latency. It takes a long time to set up and get that first message out there. After that, it’s solid.

But if you’re looking for constant speed try Infiniband (http://en.wikipedia.org/wiki/Infiniband), Myriet (www.myri.com), or QsNet (http://en.wikipedia.org/wiki/QsNet).

Gamers

Oh, those gamers. Without their demand for faster, bigger, better, where would we be?

For example, do not overlook the gaming console. NCSA (National Center for Supercomputing Applications) has a cluster of Sony PlayStations. The PlayStation 3 runs Yellow Dog Linux. The average PS3 retails for around $600. The Folding@home grid runs on PS3s and PCs.

Then we come to the GPU (Graphics Processing Unit). GPU computing means using the GPU to do general purpose scientific and engineering computing. The model for GPU computing couples a CPU with a GPU, with the GPU performing the heavy processing. (http://www.nvidia.com/object/GPU_Computing.html)

One of the hottest GPU’s is the NVIDIA Tesla GPU which is based on the CUDA GPU architecture code-named the “Fermi”.

FPGAs (Field Programmable Gate Arrays)

Technological devices keep getting smaller and smaller, and the machinery gets buried under tons of software burdened with the menu systems connected to the development environment from hell.

FPGAs take you back to the schematic level. (I was known as a “bit-twiddler” at IBM.)

My old friends at National Instruments (http://www.ni.com/fpga/) have NI LabView FPGA. LabView FPGA provides graphical programming of FPGAs.

Their video on FPGA Technology is a good intro to FPGA. (http://www.ni.com/fpga_technology/). Several other videos are available at this same sight go into further detail. For more info on the FPGA hardware see http://en.wikipedia.org/wiki/FPGA.

(I still haven’t forgiven NI for nuking my data acquisition PC with their demo. I lost a lot of stuff. All was backed up, but re-installing was not fun.)

FYI -The industry is desperately seeking parallel and FPGA programmers.

Data Representation in Database Design

The most recent programming languages are object-oriented. However, the most efficient databases are relational. There are object-oriented database systems, but for the most part they are very expensive and very, very slow. Postgres is a RDBS (Relational Database Management System) that does implement a form of inheritance where one table may extend (inherit) another table.

Then you have XML. XML Schemas are adding another dimension to this complexity. XML is popular for communication (SOAP) and representation (XSLT). Data comes from an RDMS, gets stuffed into objects, translated to XML on one end and sent as XML, translated to objects, and stored in a RDMS at the other end.

The mapping of objects to RDBMS is known as object relational (O/R) impedance mismatch. See this link for a discussion (http://www.agiledata.org./) of software development processes and a link to a recent book on database techniques for mapping objects to relational datbases – http://www.agiledata.org/essays/mappingObjects.html.

But beware, as most of these ORMs (Object to Relational Mapping) sometimes produce a schema that wouldn’t be completely relational and therefore suffer in performance. Also, the SQL produced by ORMs may not be optimal.

To effectively design and develop a RDBMS , learn UML (Universal Modeling Language). The Objects By Design web site (http://www.objectsbydesign.com) covers UML and a lot of other object-oriented topics and is worth a look.

Rational Rose is the UML design tool that I use. It’s now been purchased by IBM. Rational uses what is known as the Rational Unified method,

Speaking of XML, some of the UML design tools can now output XML directly from the data record definitions.

See this link for a list of current UML products – http://www.objectsbydesign.com/tools/umltools_byCompany.html.

The End of SQL

The ComputerWorld blog sight has an interesting 3-part series entitled – The End of SQL and relational databases?

Part 1 covers Relational Methodology and SQL.The link to part I is here – http://blogs.computerworld.com/15510/the_end_of_sql_and_relational_databases_part_1_of_3.

Part 2 is a list of current NoSQL databases. The link to part 2 is here – http://blogs.computerworld.com/15556/the_end_of_sql_and_relational_databases_part_2_of_3

Part 3 is a list of links to NoSQL sites, articles, and blog posts. The link to part 3 is here - http://blogs.computerworld.com/15641/the_end_of_sql_and_relational_databases_part_3_of_3

In short, the “NoSQL” (http://en.wikipedia.org/wiki/NoSQL) movement and cloud-based data stores are striving to completely remove developers from a reliance on SQL and relational databases.

In a post-relational world, they argue that a distributed, context-free key-value store is probably the way to go. This makes sense when are can be thousands of sequence searchers, but only one updater. A transactional database would be overkill.

Part 5 of Effective Bioinformatics Programming coming soon..

Effective Bioinformatics Programming - Part 3

Pam — Fri, 12 Feb 2010 20:46:30 +0000

All Things Unix

Bioinformatics started with Unix. At the Human Genome Center, for a long time, I had the one and only PC. (We got a request from our users for a PC-based client for the Search Launcher). Everything else was Solaris (Unix) and Mac, which was followed by Linux.

Unix supports a number of nifty commands like grep, strings, df, du, ls, etc. These commands are run inside the shell, or command line interpreter, for the operating system (Unix). There have been a number of these shells in the history of Unix development.

The bash shell http://en.wikipedia.org/wiki/Bash is the default shell for the Linux environment. This shell provides several unique capabilities over other shells. For instance, bash supports a history buffer of system commands. With the history buffer, the “up” arrow will return the previous command. The history command lets you view a history of past commands. The bang operator (!) lets you rerun a previous command from the history buffer. (Which saves a lot of typing!)

bash enables a user to redirect program output. The pipeline feature allows the user to connect a series of commands. With the pipeline (“|”) operator, a chain of commands can be linked together where the output of one command is the input to the next command an so forth.

A shell script (http://en.wikipedia.org/wiki/Shell_script) is script written for the shell or command line interpreter. Shell scripts enable batch processing. Together with the cron command, these scripts can be set to run automatically at times when system usage is minimum.

For general information about bash, go to the Bash Reference Manual at http://www.gnu.org/software/bash/manual/bashref.html .

A whole wealth of bash shell script examples is available at - http://tldp.org/LDP/abs/html/.

Unix on Other Platforms

Cygwin (http://www.cygwin.com/) is a Linux-like environment for windows. The basic download installs a minimum environment, but you can add additional packages at any time. Go to http://cygwin.com/packages/ for a list of Cygwin packages available for download.

Apple’s OS X is based on Unix. Other than the MACH kernel, the OS is BSD-derived. Their Java package is usually not the latest as Apple has to port Java due to differences such as the graphics portion.

All Things Software – Documenting and Archiving

I’ve run into all sorts of approaches to program code documentation in my career. A lead engineer demanded that every line of assembler code be documented. A senior programmer insisted that code should be self-documenting.

By that, she used variable names such as save_the_file_to_the_home_directory, and so on. Debugging these programs was a real pain. The first thing you had to do was set up aliases for all the unwieldy names.

The FORTRAN programmers cried when variable names longer than 6 characters were allowed in version 77 of VAX FORTRAN.. Personally, I thought it was great. The same with IMPLICIT NONE.

In the ancient times, FORTRAN integers variables had to start with i thru n. Real variables could use the other letters. The IMPLICIT NONE directive told the compiler to shut that off.

All FORTRAN variables had to be in capital letters. But you could stuff strings into integer variables which I found extremely useful. All FORTRAN statements had to begin with a number. This number usually started at 10 and went up in increments of 10.

At one time Microsoft used Hungarian notation (http://en.wikipedia.org/wiki/Hungarian_notation) for variables in most of their documentation. In this method, the name of the variable indicated it’s use. For example, lAccountNumber was a long integer.

The IDEs (Eclipse, NetBeans, and others) will automatically create the header comment with a list of variables. The user just adds the proper definitions. (If you’re using Java, the auto comment is JavaDoc compatible, etc.)

Otherwise, Java supports the JavaDoc tool, Python has PyDoc, and Ruby has RDoc.

Personally, I feel that software programs should be read like a book, with documentation providing the footnotes, such as an overview of what the code in question does and a definition of the main variables for both input and output. Module/Object documentation should also note who uses the function and why. Keep variable names short but descriptive and make comments meaningful.

Keep code clean, but don’t go overboard. I worked with one programmer who stated, “My code is so clean you could eat off it.” I found that a little too obnoxious, not to mention overly optimistic as a number of bugs popped out as time went by.

Archiving Code

Version Control Systems (VCS) have evolved as source code projects became larger and more complex.

RCS (Revision Control System) meant that the days of the keeping the Emacs numbered files (e.g. foo.~1~) as backups were over. RCS used the diff concept (just kept a list of the changes make to a file as a backup strategy).

I found this unsuited for what I had to do – revert to an old version in a matter of seconds.

CVS was much, much better. CVS was replaced by Subversion. But they’re centralized repository structure can create problems. You basically check out what you want to work on from a library and check it back in when you’re done. This can be a slow process depending on network usage or central server available.

The current favorite is Git. Git was created by Linus Torvalds (of Linux fame). Git is a free, open source distributed version control system. (http://git-scm.com/).

Everyone on the project has a copy of all project files complete with revision histories and tracking capabilities. Permissions allow exchanges between users and merging to a central location is fast.

The IDE’s (Eclipse and NetBeans) will have CVS and Subversion plug ins already configured for accessing those repositories. NetBeans also supports Mercurical. Plug ins for the other versioning software modules are available on the web. The Eclipse plug in for Git is available at http://git.wiki.kernel.org/index.php/EclipsePlugin.

System Backup

Always have a plan B. My plan A had IT backup my systems on a weekly to monthly basis based on usage. A natural disaster completely decimated my systems. No problem, I thought, I have system backup. Imagine how I felt when I heard that IT had not archived a single on of my systems in over three years! Well, I had a plan B. I had a mirror of the most important stuff on an old machine and other media. We were back up almost immediately.

The early Tandem NonStop systems (now known as HP Integrity NonStop) automatically mirrored your system in real-time, so down time was not a problem.

Real-time backup is expensive and unless you’re a bank or airline, it’s not necessary.

Snapshot Backup on Linux with rsync

If you’re running Linux, Mac, Solaris, or any Unix-based system, you can use rsync for generating automatic rotating “snapshot” style back-ups. These systems generally have rsync already installed. If not, the source is available at – http://rsync.samba.org/.

This website - http://www.mikerubel.org/computers/rsync_snapshots/ will tell you everything you need to know to implement rsync based backups, complete with sample scripts.

Properly configured, the method can also protect against hard disk failure, root compromises, or even back up a network of heterogeneous desktops automatically.

Acknowledgment – Thanks, Bill!

I want to thank Bill Eaton for his assistance with these blog entries on Effective Bioinformatics Programming. He filled in a lot of the technical details, performed product analysis, and gave me direction in writing these blog entries.

To Be Continued - Part 4

Part 4 will cover relational database management systems (RDBMS), HPC (high performance computing) - parallel processing, FPGC, clusters, grids, and other topics.

Effective Bioinformatics Programming - Part 2

Pam — Fri, 29 Jan 2010 20:29:35 +0000

Effective Bioinformatics Programming – Part 2

Instrumentation Programming

Instrumentation Programming usually concerns computer control over the actions of an instrument and/or the streaming or download of data from the device. Instrumentation in the Life Sciences covers data loggers, waveform data acquisition systems, pulse generators, image capture, and others used extensively in LIMS (Laboratory Information Management Systems), Spectroscopy, and other scientific arenas.

Most instruments are controlled by codes called “control codes”. These codes are usually sent or received by a C/C++ program. Some instrumentation manufacturers, however, have a proprietary programming language that must be used to “talk” to the instrument.

Some companies are nice enough to provide information on the structure of the data that comes from their instrument. When they don’t you may have to use good old “reverse engineering”. That’s where the Unix/Linux od utility comes in handy, because lots of time will be spent poring over hex dumps.

As you can tell, programming instruments requires a lot of patience. This is especially true if everything hangs or gets into a confused state. There is nothing you can do but recycle the power to everything and start over. This is usually accompanied by a banging of keyboards and the muttering of a few choice words.

Development Platforms or IDEs (Integrated Development Environment)

I have to mention development platforms as they can be useful, but also problematic. My favorite is Eclipse (http://www.eclipse.org). Originating at IBM, Eclipse was supported by a consortium of software vendors. Eclipse has now become the Eclipse open source community, supported by the Eclipse Foundation.

Eclipse is a development platform for programmers comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle. You can find plug-ins that will enable you to accomplish just about anything you want to do. A plug-in is an addition to the Eclipse platform that is not included in the base package, like an Eclipse memory manager or a debugging a Tomcat servlet.

Sun offers NetBeans (“The only IDE you need.”). I used NetBeans (http://netbeans.org) at lot on the Mac. Previously, Sun offered StudioOne and Creator. I used StudioOne (on Unix) and Creator (on Linux). I haven’t worked with NetBeans lately because they’re currently mostly Swing-centric (GUI) development and are not fully JSF (java Server Faces) aware. NetBeans will make a template for JSF but doesn’t (as yet) provide an easy way to create a JSF interface.

There are two main problems with development platforms. For one, the learning curve is fairly steep. There area lot of tutorials and examples available, but you still have take the time to do it.

The best way to use a development platform is to divide the work. One group does web content, one group does database, one group does middleware (the glue that holds everything together), etc. Each group or person can then become knowledgeable in their area and move on or absorb other areas as needed.

The second problem with these tools in that you are stuck with their developmental approach.

You have to do things a certain way and adhere to a certain structure. Flexibility can be a problem.

This is especially true of interface building. You are stuck with the code the tool generates and the files and file structures created. With most tools, you have to use that tool to access files that the tool created.

IDEs can be useful in that they will perform mundane coding tasks for you. For instance, given a database record, the IDE can use those table elements to generate web forms and the SQL queries driving those forms. You can then expand the simple framework or leave as is.

Open Source/Free Software and Bioinformatics Libraries

There a lot of good an not-so-good Open Source code out there for the Life Sciences.

There are several “gotchas” to look out for, including –

Is the code reliable? Are others using it? Are they having problems?

Will the code run on your architecture? What will it take to install

What kind of user support is available? What’s the response time?

Is there a mailing list available for the library, package, or project of interest?

The are several bioinformatics software libraries available for various languages. All of these libraries are OpenSource/Free Software. Installing these libraries takes a little more that just downloading and uncompressing a package. There are “dependencies” (other libraries, modules, programs, and access to external sites) that must be resident or accessible before a complete build of these libraries is possible.

The following is a list of the most popular libraries and their respective dependencies.

BioPerl 1.6.1: Modules section of http://www.cpan.org/

Required modules:
perl               => 5.6.1
IO::String         => 0
DB_File            => 0
Data::Stag         => 0.11
Scalar::Util       => 0
ExtUtils::Manifest => 1.52

Required modules for source build:
Test::More    => 0
Module::Build => 0.2805
Test::Harness => 2.62
CPAN          => 1.81

Recommended modules:  some of these have circular dependencies
Ace                       => 0
Algorithm::Munkres        => 0
Array::Compare            => 0
Bio::ASN1::EntrezGene     => 0
Clone                     => 0
Convert::Binary::C        => 0
Graph                     => 0
GraphViz                  => 0
HTML::Entities            => 0
HTML::HeadParser          => 3
HTTP::Request::Common     => 0
List::MoreUtils           => 0
LWP::UserAgent            => 0
Math::Random              => 0
PostScript::TextBlock     => 0
Set::Scalar               => 0
SOAP::Lite                => 0
Spreadsheet::ParseExcel   => 0
Spreadsheet::WriteExcel   => 0
Storable                  => 2.05
SVG                       => 2.26
SVG::Graph                => 0.01
Text::ParseWords          => 0
URI::Escape               => 0
XML::Parser               => 0
XML::Parser::PerlSAX      => 0
XML::SAX                  => 0.15
XML::SAX::Writer          => 0
XML::Simple               => 0
XML::Twig                 => 0
XML::Writer               => 0.4

Some of these modules such as SOAP::Lite depend upon many other
modules.

BioPython 1.53:  http://biopython.org/

Additional packages:
NumPy     (recommended) http://numpy.scipy.org/
ReportLab (optional)    http://www.reportlab.com/software/opensource/
MySQLdb   (optional)    May be in core Python distribution.

BioRuby 1.4.0:  http://www.bioruby.org/

The base distribution is self-contained and uses the RubyGems installer.
Optional packages.

RAA:xmlparser
RAA:bdb
RubyForge:ActiveRecord and at least one driver (or adapter) from
   RubyForge:MySQL/Ruby, RubyForge:postgres-pr, or RubyForge:ActiveRecord
   Oracle enhanced adapter.
RubyForge:libxml-ruby (Ruby language bindings for the GNOME Libxml2 XML toolkit)

BioJava 1.7.1:  http://www.biojava.org/

biojava-1.7.1-all.jar:  self-contained binary distribution with
  all dependencies included.

biojava-1.7.1.jar:  bare distribution that requires the following additional
  jar files.  These are required for building from source code.
  Most are from http://www.apache.org/

bytecode.jar:                  required to run BioJava
commons-cli.jar:               used by some demos.
commons-collections-2.1.jar:   demos, BioSQL Access
commons-dbcp-1.1.jar:          legacy BioSQL access
commons-pool-1.1.jar:          legacy BioSQL access
jgraph-jdk1.5.jar:          NEXUS file parsing

Don’t forget to sign up for the mailing list for that library or libraries of interest to get the lastest news, problems, solutions, etc. for that library or just life science topics in general.

Software Hosting and Indexing Sites

There are several Software Hosting and Indexing Sites that serve as software distribution points for bioinformatics software.

SourceForge.net – Search on bioinformatics for a list of software available. Projects include:MIAMExpress - http://sourceforge.net/projects/miamexpress/

freshmeat– The Web’s largest index of Unix and cross-platform software

Bioinformatics Organization – The Open Access Institute

Open Bioinformatics Foundation (O|B|F) - Hosts Many Open Bioinformatics Projects

Public Domain Manifesto

In this time of curtailment of civil rights, the Public Domain Manifesto seems appropriate (http://www.publicdomainmanifesto.org/node/8 ). Sign the petition while you’re there.

This is the end of Part 2. Part 3 will explore more software skills, project management, and other computational topics.

Effective Bioinformatics Programming - Part 1

Pam — Thu, 21 Jan 2010 21:38:42 +0000

The PLOS Computational Biology website recently published “A Quick Guide for Developing Effective Bioinformatics Programming Skills” by Joel T. Dudley and Atul J. Butte (http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000589 ).

This article is a good that survey covers all the latest topics and mentions all the currently-popular buzzwords circulating above, around, and through the computing ionosphere. It’s a good article, but I can envision readers’ eyes glazing over about page 3. It’s a lot of computer-speak in a little space.

I’ll add in a few things they skipped or merely skimmed over to give a better overview of what’s out there and how it pertains to bioinformatics.

They state that a biologist should put together a Technology Toolbox. They continue, “The most fundamental and versatile tools in your technology toolbox are programming languages.”

Programming Concepts

Programming languages are important, but I think that Programming Concepts are way, way more important. A good grasp of programming concepts will enable you to understand any programming language.

To get a good handle on programming concepts, I recommend at book. This book, Structure and Implementation of Computer Programs from MIT Press (http://mitpress.mit.edu/sicp/),is the basis for an intro to computer science at MIT. It’s called the Wizard Book or the Purple Book.

I got the 1984 version of the book which used the LISP language. The current 1996 version is based on LISP/Scheme. Scheme is basically a cleaned-up LISP, in case you’re interested.

Best of all course (and the down loadable book) are freely available from MIT through the MIT OpenCourseWare website – http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-001Spring-2005/CourseHome/index.htm.

There’s a blog entry - http://onlamp.com/pub/wlg/8397 - that goes into further explanation about the course and the book..

And just because you can program, it doesn’t mean you know (or even need to know) all the concepts. For instance, my partner for a engineering education extension course was an electrical engineer who was programming microprocessors. When the instructor mentioned the term “scope” in reference to some topic, he turned to me and asked, “What’s scope?”

According to MIT’s purple book –” In a procedure definition, the bound variables declared as the formal parameters of the procedure have the body of the procedure as their scope.”

You don’t need to know about scope to program in assembler, because everything you need is right there. (In case you’re wondering, I consider assembler programmers to be among the programming elites.)

Programming Languages

The article mentions Perl, Python, and Ruby as the “preferred and most prudent choices” in which to seek mastery for bioinformatics.

These languages are selected because “they simplify the programming process by obviating the need to manage many lower level details of program execution (e.g. memory management), affording the programmer the ability to focus foremost on application logic…”

Let me add the following. There are differences in programming languages. By that, I mean compiled vs scripted. Languages such as C, C++, and Fortran are compiled. Program instructions written in these languages are parsed and translated into object code, or a language specific to the computer architecture the code is to run on. Compiled code has a definite speed advantage, but if the code is the main or any supporting module is changed, the entire project must be recompiled. Since the program is compiled into the machine code of a specific computer architecture, portability of the code is limited.

Perl, Python, and Ruby are examples of scripted or interpreted languages. These languages are translated into byte code which is optimized and compressed, but is not machine code. This byte code is then interpreted by a virtual machine (or byte code interpreter) usually written in C.

An interpreted program runs more slowly than a compiled program. Every line of an interpreted program must be analyzed as it is read. But the code isn’t particularly tied to one machine architecture making portability easier (provided the byte code interpreter is present). Since code is only interpreted at run time, extensions and modifications to the code base is easier, making these languages great for beginning programmers or rapid prototyping.

But, let’s get back to the memory management. This, and processing speed will be a huge deal in next gen data analysis and management.

Perl automatic memory management has a problem with circularity, as Perl (and Ruby and Python) count references.

If object 1 points to object 2 and object 2 points back to 1 , but nothing else in the program points to either object 1 or object 2 (this is a weak reference), these objects don’t get destroyed. They remain in memory. If these objects get created again and again, it’s called a memory leak.

I also have to ask – What about C/C++ , Fortran, and even Turbo Pascal? The NCBI Toolkit is written in C/C++. If you work with foreign scientists, you will probably see a lot Fortran.

Debugging

You can’t mention programming with mentioning debugging. I consider the act of debugging code an art form any serious programmer should doggedly pursue.

Here’s a link to a ebook, The Art of Debugging – http://www.circlemud.org/cdp/hacker/. It’s mainly Unix-based, C-centric and a little dated. But good stuff never goes out of style.

Chapter 4, Debugging: Theory explains various debugging techniques. Chapter 5 – Profiling talks about profiling your code, or determining where your program is spending most of its time.

He also mentions core dumps. A core is what happens when your C/C++/Fortran program crashes in Unix/Linux. You can examine this core to determine where your program went wrong. (It gives you a place to start.)

The Linux Foundation Developer Network has an on-line tutorial – Zen and the Art of Debugging C/C++ in Linux with GDB – http://ldn.linuxfoundation.org/article/zen-and-art-debugging-cc-linux-with-gdb. They write a C program (incorporating a bug), create a make file, compile, and then use gdb to find the problem. You are also introduced to several Unix/Linux commands in the process.

You can debug Perl by invoking it with the -d switch. Perl usually crashes at the line number that caused the problem and some explanation of what went wrong.

The -d option also turns on parser debugging output for Python.

Object Dumps

One of the most useful utilities in Unix/Linux is od (object dump). You can examine files in octal (default), hex, or ASCII characters

od is very handy for examining data structures, finding hidden characters, and reverse engineering.

If you think you’re code is right, the problem may be in what you are trying to read. Use od to get a good look at the input data.

That’s it for Part 1. Part 2 will cover Open Source, project management, archiving source code and other topics.

Keep to Good Faith

Pam — Wed, 11 Nov 2009 20:28:43 +0000

I was going through a box of textbooks last week and stumbled upon a copy of the Enron Code of Ethics. I have another one stored away with a form, signed by Ken Lay, that states I have read and will comply with the Enron Code of Ethics.

I was employed at Enron from 2000 through 2002 and was there when the wheels came off. Our department was left intact. Otherwise, whole floors of the Enron building were vacated. It really was a shame, because Enron was a great place to work. Several friends and acquaintances lost most of what they had because of the malfeasance of a greedy few.

This had to be the most blatant example of unethical conduct in the workplace I encountered. There were others, that appeared seemingly minor, ended up costing companies money and talent. Most of these losses were mostly the result of mismanagement and not outright unethical behavior. But, then again, is mismanagement itself unethical?

I book I read recently entitled “A Small Treatise of the Great Virtues, The Uses of Philosophy in Everyday Life” by Andre Comte-Sponville (Metropolitan Books), talks about truth as “Good Faith”.

He states on page 196, that “at the very least that one speaks the truth about what one believes, and this truth, even if what one believes is false, is less true for all that. Good faith, in this sense, is what we cal sincerity (or truthfulness or candor) and is the opposite of mendacity, hypocrisy, and duplicity, in short, the opposite of bad faith in all its private and public forms.”

In my position at a major hardware/software developer I was told that I “didn’t need to know about a product to sell it.”

At another position, I found that a few fraudulent claims by a contractor caused a company to fork over three quarters of a million dollars for custom software when a fifteen thousand dollar piece of hardware would have an enabled an already existing piece of commercial software to do the job. With a more accurate accountability of the data, I might add.

In fact, the whole program was completely mismanaged, to the detriment of the company, not the contractor. In fact, he was ready for the next program as he had one of his engineers hired in to head up that project. An engineer who didn’t have the slightest idea about our system, much less its theory. Thankfully, we got him transferred out of there and back to design where he belonged. The contractor was kicked out of the company.

These are straight-forward examples of bad faith. The following are a little harder to classify.

Beware the ulterior motive, especially if the new system you are proposing will impose on someone’s fiefdom.

Data analysis for the existing program consisted of placing a request with the a data analysis group and waiting up to 3 days for results. The system proposed (and later deployed) would give each and every engineer access to an analysis application that they could use to inspect the data one and a half hours after a particular test cycle was completed. A little training and they were ready to go.

Countless hours were spent in useless meetings defending the system. Everybody shut up when the system came up on day one and stayed up through months of testing.

This test/record/analysis cycle fits perfectly into the Laboratory Information Management Systems (LIMS) cycle of genomics research. A successful LIMS implementation in one lab aroused the ire of yet another lab attempting to develop their own solution. Let’s just say a lot of bad faith erupted.

The real loser in the above examples is the company. Money is wasted and talented people go elsewhere.

Biotechnology is a hot commodity right now. Stimulus funding bringing fresh capital to many projects. Companies are leveraging existing corporate products by repackaging them as biotech ready.

National Instruments LabView is one of these. I used it a lot in engineering. Now it’s a big player in the lab, incorporating interfaces for research lab instrumentation.

What is a LIMS (Laboratory Information Management System)? Is it an inventory management system? Is it a data pipeline? Can one size fit all?

Some companies have taken existing Inventory Management Systems and relabeled them as a Laboratory Information Management Systems. (At least the acronym fits.) Most of these systems don’t distinguish between research and manufacturing environments. They also don’t support basic validation of the LIMS application for its intended purpose. No wonder some 80% of LIMS users are dissatisfied.

At a recent conference I talked with researchers from various pharmaceutical companies and they were thoroughly dissatisfied with their LIMS systems. One scientist stated that they had a problem with their LIMS. When they went to report the problem, they found the company was no longer in business.

The latest IT (Information Technology) trends – SaaS, Cloud computing – may work in a business environment , but they won’t translate well to a pharmaceutical research area where they want everything safe behind the firewall.

There are many, many factors that go into developing biotechnology applications. Getting the right people, controlling the political environment, finding or developing the right software – it’s a jungle out there.

Keep to Good Faith and please be careful.

Women In Technology

Pam — Wed, 30 Sep 2009 17:29:35 +0000

Today, one in ten engineers is a woman – http://www.dol.gov/wb/factsheets/hitech02.htm In avionics, it’s fewer than that.

This is really a shame, because I find that women are extremely well suited for jobs in high tech careers.

Here’s a short list of why I think this is true along with explanations as to why I think this is so.

Women are more patient and determined
Women can juggle a lot of tasks simultaneously
Women can attend to small details and see the big picture at the same time
Women don’t get derailed by the small stuff
Women have a better support system.
Women are more sympathetic and understanding

I’ll stop at this group of six, although I could add a few more. They are not true of all women, but that’s probably because they haven’t had the experience.

Just take a look at what current society expects of women and I think you’ll see why I think women are more patient and determined! Case in point, I just got an email on “How to Create Perfect Eyes” through makeup application. Can you imagine a heterosexual male having the patience to take the time to apply all the goop we women have to put on our faces to be seen in public? Also, remember how determined we were to walk in high heels so we could pretend we were grown-ups?

Programming, system design and integration requires patience and determination. It’s a step-by-step process. All the pieces have to work together to produce the correct outcome. It’s no different that making a food dish from a recipe, although in most cases you’ll have only your experience to formulate the list of ingredients and right steps to finish the job.

Think about getting the family ready for school/work in the morning. How many things are you trying to do at once? Multi-tasking is standard operating procedure for most women, who can adapt to chaos in the blink of an eye.

I know chaos. Other than being the oldest of nine children (5 girls, 4 boys), I drove a school bus for about 4 years while I was attending college. I was given a long, country route that paid well and gave me enough hours to qualify for health insurance. After I had driven the route for about six weeks, my supervisor asked me how i was doing and what did I think of the kids. I said I thought I was doing okay and the kids were a little rowdy, but we got that under control. Otherwise, I said the kids were a bright bunch and generally inquisitive about everything. (“Miss Pam, what’s a hickey? Our teacher says it’s something you get in dominoes.”)

I found out later that these kids had been through 4 bus drivers in 4 weeks. The last day of that school year the kids on the route gave me a plaque that said “World’s Best School Bus Driver”. I was impressed, even though they misspelled my name.

I’ve discovered that women, as a whole, performed better on mission critical tasks that required a lot on concentration and coordination of several activities that had to occur simultaneously.

I couldn’t make a practice session for a particular field test, so the guys were going to fill in for me. I heard that it took them an extra long time to get started, because they couldn’t figure out how to calibrate the instrumentation. (They took the same training class that I did!) Let’s just say that they were more than happy to let me take over the operation after they were introduced to all the steps involved in the pre and post fly-over operations.

Lots of tasks mean lots of details to keep track of with almost no time to double-check anything. Women do this sort of thing all the time. Think about putting together a meal, folding clothes fresh from the dryer, putting on makeup. You don’t really think about it, you just do it. Juggling home, family, and career by itself is one big accomplishment.

We took two years to perfect all the pieces that made up the testing for the 727QF certification. We worked out the weather station in Roswell, NM. We took the acoustic analyzer to Moses Lake, WA ( to work out the routine we needed for testing. (Desert dust at 35 knots in no fun, but it can’t hold a candle to the volcanic ash from Mt. St. Helen’s that we ran into in Moses Lake. They got about a foot of ash from that explosion and the ash was dumped at the airport. Right where we were working!)

The only missing piece was the data download from the data logger on the meteorological (met) plane.

I sat under the wing of the small Cessna in the hot Texas August heat with a laptop atop my crossed legs, dodging fire ants, as I worked out the best method for our technician to save the data acquired after each run of the met plane. I got it down to a few steps, ran through it with him, and we had the met data canned.

All those pieces, met plane, weather station, acoustic analyzer and DAT (digital audio tape) data, were part of the big picture that was noise testing. The other parts were the group support systems – data download, availability, and analysis, There was so much data flowing through the pipeline, we held a meeting every morning to discuss who needed what, how much, how they wanted it, and what data could be taken to archive.

The next-gen sequencing efforts are producing an astronomical amount of raw data. Data that has to be stored, analyzed, and archived, creating one complex system. It’s a massive task and one I can sympathize with.

Women don’t get derailed by the small stuff.

Maybe this wasn’t so small, and sometimes it hit close to home, but a lot of the things I did got satirized via a cartoon or paste-up on bulletin boards all over the plant on the 727QF program.

For instance, I developed this relational database model that would store measurement information for the two aircraft we were testing.

One of the technicians had started his own local database, but he had no understanding of relational data concepts. So he had thermocoupleA and thermocoupleB, where A represented on aircraft and B represented the other. The thermocouple in question was the same on both aircraft, causing duplicate records for the same part info.

At a informal meeting we were having in the instrumentation lab, I said that his database design was stupid because we didn’t need more than one copy of the part’s basic attributes. The next day there was a flyer on the bulletin boards with a picture of the tech with a bubble over his head that said, “I stupid.”

There was some other verbiage, “Coming soon son of stupid. When relational is not enough.”

Stupid Database Flyer

Since the technician was a friend, this was funny. There were others that weren’t so entertaining.

I think women are more sympathetic and understanding of other people. The problem is to not be so understanding that you are taken for a ride.

As a support system, we have probably the best weapon in the arsenal – we can cry. Not in public, not on the job, but we can got somewhere private and cry. Sometimes this is the only way to get it our of your system.

I put a lot of dents in a lot of old hardware and ran miles and miles, but. sometimes. even that did not cover it.

I will end by saying that I was pleasantly surprised at the number of women involved in the life sciences. By this, I mean as directors, P.I.’s, or other positions of power. However, men in the field still earn one-third more than the women.

Maybe one day, women will wield as much power in all branches of technology, and their paychecks will actually reflect this status.

BioCamp 2009 at Rice University

Bill and I attended BioCamp 2009 at Rice University on Saturday, Sept. 12. There were several presentations followed by lively question and answer sessions.

The atttendance consisted of entrepreneurs, those seeking guidance on turning their ideas and research into viable products, consultants searching for marketable products, and members of the legal profession offering advice on intellectual property, patents, trademarks, and the like.

Science Communication

Pam — Fri, 10 Jul 2009 18:48:28 +0000

A Commentary piece, “Science Communication Revisited”, in the June, 2009, issue of Nature Biotechnology discusses increasing public involvement in science issues and decision-making.

Concerns are raised about the state of science education and scientific literacy more generally.

If only the public were more knowledgeable about things scientific, the article states, they would see things through the eyes of the expert.

Education

I was fortunate enough to have attended private schools from elementary through high school. Very few children are so lucky.

My biology teacher was a Catholic nun. She introduced us to Teilhard de Chardin (http://en.wikipedia.org/wiki/Pierre_Teilhard_de_Chardin).

Teilhard was a Jesuit priest who was trained as a paleontologist and geologist and took part in the discovery of Peking Man. He also studied botany and zoology. His book, The Phenomenon of Man, talks about the unfolding of the material cosmos towards the Omega Point, a maximum level of consciousness, that is pulling all creation towards it. Evolution, according to Teilhard, was the process of matter becoming aware of itself.

Therefore, I was able to receive a fairly sound exposure to evolution. On the other hand, the chapters on male and female biology and the reproductive process was ripped out of my text book.

(I know, because we found an unaltered book and read that forbidden text.)

At any rate, I grew up in an agricultural environment and knew what it was all about.

If you’re interested in the state of scientific education or education in public schools in Texas, I recommend the Texas Freedom Network (http://txfree.convio.net/site/PageServer ).

Experts

Concerning experts, I remember my section chief telling me, “You have to forgive Bryan, he still believes in experts.” Brian was our lead engineer.

As far as experts go, you have to be able to separate the good from the bad.

I recommend this article, Crap Detection 101 (http://www.sfgate.com/cgi-bin/blogs/rheingold/detail?entry_id=42805) and the CRAP Test (http://www.workliteracy.com/the-crap-test).

The CRAP test is a way to evaluate an internet source based on the following criteria: Currency, Reliability, Authority and Purpose/Point of View.

The article and test’s main focus is the internet — how to tell real from bogus. It’s not too hard to extrapolate the points they make to everyday life.

Scientific Literacy

Science and technology are changing so rapidly, that many people have simply given up on trying to keep up. Their scientific literacy consists of newspaper articles or blurbs on the TV news.

A lot of what is presented as science on network television is implausible (not to mention the technology used on these shows).

I think to really succeed, real scientists must pay attention to what is presented to the general public and critique it through publications, such as letters to the editor, blogs, appearances, etc. as much as possible.

Scientists should also by of an open mind as to the intelligence of your audience.

We have way too many people with 200 point IQ levels digging ditches in this country. We spend an inordinate amount of funds and interest on educating special children. We should be spending just as much time and funds (if not more) identifying and encouraging the geniuses among us who find education boring and quickly loose interest.

The interest in science is out there, but scientists must take an interest in how what passs for science is disseminated, validate or invalidate that science, identify the appropriate target audience, and address that audience level to really open up the forum on true scientific communication.

Salon, an e-zine, has a really good article. Why America is flunking science (http://www.salon.com/env/feature/2009/07/13/science_illiteracy/?source=newsletter) that is worth the read.

Here’s another link where the author lists current “myths” surrounding scientists engaging with the media.

http://scienceblogs.com/christinaslisrant/2009/07/when_discussing_scientists_eng.php