Kernel

Basicintroduction

Thekernelisthecoreofanoperatingsystem.Itisthefirst-layersoftwareexpansionbasedonhardware,whichprovidesthemostbasicfunctionsoftheoperatingsystem.Itisthebasisfortheworkoftheoperatingsystem.Itisresponsibleformanagingthesystem’sprocesses,memory,devicedrivers,filesandnetworksystems,anddeterminestheperformanceandperformanceofthesystem.stability.

Inthemodernoperatingsystemdesign,inordertoreducetheoverheadofthesystemitself,itisoftencloselyrelatedtothehardware(suchasinterrupthandlers,devicedrivers,etc.),basic,common,andhigheroperatingfrequency.Themodules(suchasclockmanagement,processscheduling,etc.)andkeydatastructuresareseparatedtomakethempermanentinmemoryandprotectthem.Usuallythispartiscalledthekerneloftheoperatingsystem.

Programscanbedirectlytransferredtothecomputerforexecution.Thisdesignshowsthatthedesignerdoesnotwanttoprovideanyhardwareabstractionandoperatingsystemsupport.Itiscommoninthedesignofearlycomputersystems.Finally,someauxiliaryprograms,suchasprogramloaderanddebugger,aredesignedintothecoreofthemachine,orsolidifiedinread-onlymemory.Whenthesechangesoccurred,theconceptoftheoperatingsystemkernelgraduallybecameclear.

(Summarypicturesource:)

Historicaldevelopment

ThefirstpublicversionofLinuxwasversion0.02inOctober1991.Twomonthslater,InDecember1991,Linuxreleasedversion0.11,whichwasthefirstindependentkernelthatcouldbeusedwithoutrelyingonMinix.

Onemonthafterthereleaseofversion0.12,inMarch,theversionnumberjumpedto0.95,reflectingthatthesystemwasbecomingmature.Notonlythat,itwasamilestoneuntiltwoyearslater,inMarch1994.1.0.0wascompleted.

Fromthenon,westartedtousethetwo-waynumberingmethodtomarkthedevelopmentofkernels.Even-numberedkernels(suchas1.0,2.2,2.4,2.6)arestable,andthe"product"model,atthesametime,Theodd-numberedkernelversions(1.1,2.3)arecutting-edgeor"developing"kernels.Afewmonthsafterthereleaseofastablekernel,thedevelopmentofanewkernelwillbegin.However,thedevelopmentof2.5didnotstartuntildozensofmonthsafterthecompletionof2.4.

Mostofthediscussioncontentofthepost-halloweendocumentisthemainchangesthatusersneedtopayattentionto,andthesystemtoolsthatneedtobeupdated(inordertotakeadvantageofthem).ThosewhocareaboutthisinformationaremainlyLinuxdistributorswhowanttoknowinadvancewhatisinthe2.6kernel,aswellasendusers,whichallowsthemtodeterminewhetherthereareprogramsthatneedtobeupgradedinordertotakeadvantageofthenewcomponents.

TheKernelJanitorsprojectmaintainsalistofminordefectsthatneedtobefixedandsolutions.Mostofthesedefectsolutionsarecausedbytheneedtochangemanypartsofthecodewhenapplyinglargerpatchestothekernel,forexample,someplaceswillaffectthedevicedriver.Thosewhoarenewlyengagedinkerneldevelopmentcanchoosetheitemsinthelistatthebeginningoftheirwork,sothattheycanlearnhowtowritekernelcodethroughsmallprojectsandhavetheopportunitytocontributetothecommunity.

Also,inanotherpre-releaseproject,JohnCherrytrackedtheerrorsandwarningsfoundwhencompilingeachreleasedkernelversion.Thesecompilationstatisticshavebeendecliningovertime,andpublishingtheseresultsinasystematicformmakestheprogressachievedataglance.Inmanycases,someofthesewarningsanderrormessagescanbeusedliketheKernelJanitorslist,becausecompilationerrorsareusuallycausedbysmalldefectsandrequiresomeefforttofix.

Finally,thereisAndrewMorton's"must-fix"list.Sincehehasbeenselectedasthemaintainerofthe2.6kernelaftertherelease,heusedhisprivilegestooutlinetheissuesthathebelievesmosturgentlyneedtoberesolvedbeforethefinal2.6kernelisreleased.Themust-fixlistcontainsthedefectsinthekernelBugzillasystem,thepartsthatneedtobecompleted,andotherknownissues.Iftheseissuesarenotresolved,itwillhinderthereleaseof2.6.Thisinformationcanhelpindicatewhatstepsareneededbeforethenewkernelisreleased;itcanalsoprovidevaluableinformationforthosewhoareconcernedaboutwhenthemuch-anticipated2.6kernelreleasewillbecompleted.

Kernelclassification

Singlekernel

Monolithickernelisaverybigprocess.Itsinteriorcanbedividedintoseveralmodules(orhierarchicalorother).Butwhenitisrunning,itisasinglebigbinaryimage.Thecommunicationbetweenthemodulesisachievedbydirectlycallingfunctionsinothermodules,ratherthanmessagetransmission.

Thesingle-corestructuredefinesahigh-levelabstractinterfaceontopofthehardware,andusesasetofprimitives(orsystemcalls)toimplementthefunctionsoftheoperatingsystem,suchasprocessmanagement,filesystem,andstorageManagementandsoon,thesefunctionsarecompletedbymultiplemodulesrunninginthecorestate.

Althougheachmoduleservestheseoperationsseparately,thekernelcodeishighlyintegratedanddifficulttowritecorrectly.Becauseallmodulesarerunningonthesamekernelspace,asmallbugcancrashtheentiresystem.However,ifthedevelopmentgoeswell,thesingle-corestructurecanbenefitfromoperatingefficiency.

Manymodernsingle-kernelstructurekernels,suchasLinuxandFreeBSDkernels,cancallmodulesintoexecutionatruntime,whichcanmakeiteasiertoexpandthefunctionsofthekernel,anditcanalsomakethecoreofthekernel.Partbecomesmoreconcise.

Single-corestructureisaveryattractivedesign,duetothecomplexityofthesystemcontrolcodethatimplementsalllow-leveloperationsinthesameaddressspace,andtheefficiencywillbehigherthanthatindifferentaddressspacessome.Thesingle-corestructureistendingtobeeasytodesigncorrectly,soitsdevelopmentwillbefasterthanthemicro-kernelstructure.

Anexampleofasingle-kernelstructure:thetraditionalUNIXkernel-suchastheversionreleasedbytheUniversityofBerkeley,theLinuxkernel.

Microkernel

Themicrokernel(Microkernelkernel)structureconsistsofaverysimplehardwareabstractionlayerandasetofkeyprimitivesorsystemcalls.TheseprimitivesonlyincludetheestablishmentofThenecessarypartsofasystem,suchasthreadmanagement,addressspace,andinter-processcommunication.

Thegoalofmicrokernelistoseparatetherealizationofsystemservicesfromthebasicoperatingrulesofthesystem.Forexample,theinput/outputlockingserviceoftheprocesscanbeprovidedbyaservicecomponentrunningoutsidethemicrokernel.Theseverymodularuser-modeserversareusedtocompletemoreadvancedoperationsintheoperatingsystem.Thisdesignmakesthedesignofthecorepartofthekerneleasier.Thefailureofaservicecomponentwillnotcausetheentiresystemtocrash.Whatthekernelneedstodoistorestartthiscomponentwithoutaffectingotherparts.

ThemicrokernelputsmanyOSservicesintoseparateprocesses,Suchasfilesystem,devicedriver,andtheprocesscallsOSservicesthroughmessagepassing.Themicro-kernelstructuremustbemulti-threaded.Thefirst-generationmicro-kernelprovidesmoreservicesinthecore,soitiscalledthe"fatmicro-kernel",anditstypicalrepresentativeisMACH.ItisbothGNUHURDandthecoreofAPPLESERVEROS.Itcanbesaidtobethriving.ThesecondgenerationprovidesonlythemostbasicOSservicesforthemicrokernel.ThetypicalOSisQNX.QNXiswell-knowninthetheoreticalworldandisconsideredanadvancedOS..

Themicrokernelonlyprovidesasmallpartofthehardwareabstraction,andmostofthefunctionsareperformedbyaspecialuser-modeprogram:theserver.Micronucleiareoftenusedintheembeddeddesignofrobotsandmedicaldevices,becausethekeypartsoftheirsystemsareinseparate,protectedstoragespaces.Thisisimpossibleforasingle-coredesign,evenifitusesthewayofloadingmodulesatruntime.

Examplesofmicrokernels:AIX,BeOS,L4microkernelseries,.MachisusedinGNUHurdandMacOSX,Minix,MorphOS,QNX,RadioOS,VSTa.

Hybridkernel

Thehybridkernelisverysimilartothemicro-kernelstructure,exceptthatitscomponentsrunmoreinthecorestatetoobtainfasterexecutionspeed.

Thehybridkernelisessentiallyamicrokernel,butitallowssomemicrokernelstructuretorunintheuserspacecodetoruninthekernelspace,whichmakesthekernelmoreefficient.Thisisacompromise.Thedesignerreferstothetheorythatthemicrokernelstructureofthesystemrunspoorly.However,laterexperimentsprovedthatapuremicrokernelsystemcanactuallybehighlyefficient.Mostmodernoperatingsystemsfollowthisdesigncategory,andtheWindowsoperatingsystemdevelopedbyMicrosoftisagoodexample.ThereisalsoXNU,thekernelrunningonApple'sMacOSX,whichisalsoahybridkernel.

Examplesofmixedkernels:BeOSkernel,DragonFlyBSD,ReactOSkernel

WindowsNT,Windows2000,WindowsXP,WindowsServer2003andWindowsVistaandotheroperatingsystemsbasedonNTtechnology.

Outercore

Outercoresystem,alsoknownasverticalstructureoperatingsystem,isamoreextremedesignmethod.

Theexternalkerneldoesnotprovideanyhardwareabstractoperations,butallowsadditionalruntimelibrariestobeaddedtothekernel,throughwhichapplicationscandirectlyornearlydirectlyoperateonthehardware.

Itsdesignconceptistoletthedesigneroftheuserprogramdecidethedesignofthehardwareinterface.Theouterkernelitselfisverysmall,anditisusuallyonlyresponsibleforservicesrelatedtosystemprotectionandsystemresourcereuse.

Traditionalkerneldesigns(includingsingle-coreandmicro-core)haveabstractedthehardware,hidinghardwareresourcesordevicedriversunderthehardwareabstractionlayer.Forexample,inthesesystems,ifapieceofphysicalstorageisallocated,theapplicationdoesnotknowitsactuallocation.

Thegoaloftheexternalcoreistoallowtheapplicationtodirectlyrequestaspecificphysicalspace,aspecificdiskblock,andsoon.Thesystemitselfonlyguaranteesthattherequestedresourceiscurrentlyfree,andtheapplicationallowsdirectaccesstoit.Sincetheexternalcoresystemonlyprovidesrelativelylow-levelhardwareoperations,anddoesnotprovidehigh-levelhardwareabstractionlikeothersystems,thenadditionalruntimelibrarysupportisneeded.Theseruntimelibrariesrunontheoutercoreandprovidecompletefunctionsforuserprograms.

Intheory,thisdesignallowsvariousoperatingsystemstorunonanexternalcore,suchasWindowsandUnix.Andthedesignercanadjustthefunctionsofeachpartofthesystemaccordingtotheoperatingefficiency.

Theoutercoredesignisstillintheresearchstage,andnocommercialsystemhasadoptedthisdesign.Severalconceptualoperatingsystemsarebeingdeveloped,suchasNemesisfromtheUniversityofCambridge,theCitrixsystemfromtheUniversityofGlasgow,andasystemfromtheSwissAcademyofComputerSciences.MITisalsoconductingsuchresearch.

Comparisonofsingle-coreandmicro-kernel

Single-corestructureisaveryattractivedesign,becauseallcomplexlow-leveloperatingsystemcontrolsareimplementedinthesameaddressspaceTheefficiencyofthecodewillbehigherthantheimplementationinadifferentaddressspace.

Intheearly1990s,thesingle-corearchitecturewasconsideredobsolete.ThedesignofLinuxasasingle-kernelstructureinsteadofamicrokernelhascausednumerouscontroversies.

Single-corestructuretendstobelesserror-proneindesign,soitsdevelopmentwillbefasterthanmicro-kernelstructure.Therearesuccessfulcasesinbothcamps.

AlthoughMachisawell-knownmulti-purposemicrokernel,peoplehavedevelopedseveralothermicrokernels.L3isademonstrativekernel,justtoprovethatthemicro-kerneldesignisnotalwayslowspeed.ItssubsequentversionL4canevenusetheLinuxkernelasoneofitsprocesses,runninginaseparateaddressspace.

QNXisamicrokernelsystemthathasbeendesignedsincethe1980s.ItisclosertotheconceptofmicrokernelsthanMach.Itisusedinsomespecialfields;inthesecases,systemfailureduetosoftwareerrorsisnotallowed.Forexample,themanipulatoronthespaceshuttle,andthemachinethatgrindsthelensofthetelescope,alittlemistakecancausethousandsofdollarsinlosses.

ManypeoplebelievethatbecauseMachcannotsolvesomeoftheproblemsthatwereaddressedwhenthemicrokerneltheorywasputforward,themicrokerneltechnologyisuseless.FansofMachindicatethatthisisaverynarrowview,butunfortunatelyitseemsthateveryoneisbeginningtoacceptthisview.

Advantages

Abstracthiding

Thekernelprovidesahardwareabstractionmethodtocompletehardwareoperations,becausetheseoperationsareverycomplex,andhardwareabstractionhidesComplexityprovidesasetofconciseandunifiedinterfacesforapplicationsoftwareandhardware,makingprogramdesigneasier.

SourceCodeManagement

Inhistory,therehasneverbeenaformalsourcecodemanagementorrevisioncontrolsystemfortheLinuxkernel.Infact,manydevelopershaveimplementedtheirownmodifiedcontrollers,butthereisnoofficialLinuxCVSarchivethatallowsLinusTorvaldstochecktheaddedcodeandallowotherstoobtainthecode.Fixingthelackofcontrollersoftencreatesa"generationgap"betweenreleaseversions.Noonereallyknowswhatchangeshavebeenadded,whetherthesechangescanbewellintegrated,orwhatnewcontentisworthlookingforwardtointheupcomingrelease..Generally,ifmoredeveloperscanunderstandthosechangesastheyunderstandthechangestheyhavemade,certainproblemscanbeavoided.

Itisverynecessarytouseareal-time,centralizedarchivetosavethelatestupdatestotheLinuxkernel.Everychangeorpatchacceptedbythekernelistrackedasachangeset.Endusersanddeveloperscansavetheirownsourcefilearchivesandupdatethemwiththelatestsetofchangeswithasimplecommandasneeded.Fordevelopers,thismeansthatyoucanalwaysusethelatestcopyofthecode.Testerscanusetheselogicalchangesetstodeterminewhichchangescausedtheproblemandshortenthetimerequiredfordebugging.Evenuserswhowishtousethelatestkernelcandirectlyusethereal-time,centralizedarchive,becauseoncethepartsordefectfixestheyneedareaddedtothekernel,theycanupdateitimmediately.Whenthecodeisintegratedintothekernel,anyusercanprovideimmediatefeedbackanddefectreportsonthecode.

Paralleldevelopment

Kernelstructurediagram(6photos)

AstheLinuxkernelgrows,itbecomesmorecomplexandattractsmoreDevelopersfocusedtheirattentiononthespecializeddevelopmentofspecificaspectsofthekernel,andtherewasanotherinterestingchangeinthewayLinuxwasdeveloped.Duringthedevelopmentofthe2.3kernelversion,inadditiontothemainkerneltreereleasedbyLinusTorvalds,therearesomeotherkerneltrees.

Duringthedevelopmentof2.5,thekerneltreeexploded.Sincetheuseofsourcecodemanagementtoolscanmaintainthesynchronizationandparalleldevelopmentofdevelopment,itispossibletoachievepartialparallelizationofdevelopment.Inorderforotherstotesttheirchangesbeforetheyareaccepted,somedevelopmentneedstobeparallelized.Thosekernelmaintainerswhomaintaintheirowntreesworkonspecificcomponentsandgoals,suchasmemorymanagement,NUMAcomponents,improvedscalability,andcodeforspecificarchitectures,andsometreescollectandtrackcorrectionsformanysmalldefects.

Theadvantageofthisparalleldevelopmentmodelisthatitallowsdeveloperswhoneedtomakemajorchanges,orthosewhomakealargenumberofsimilarchangesforaspecificgoal,tofreelydevelopinacontrolledenvironment,Anddoesnotaffectthestabilityofthekernelusedbyothers.Whenthedevelopershavefinishedtheirwork,theycanreleasepatchesforthecurrentversionoftheLinuxkerneltoimplementthechangestheyhavedonesofar.Inthisway,testersinthecommunitycaneasilytestthesechangesandprovidefeedback.Aftereachparthasbeenproventobestable,thosepartscanbeintegratedintothemainLinuxkernelindividually,orevenallatthesametime.

(Sourceofsomepicturesinthealbumofthekernelstructurediagram:)

CodeCoverageAnalysis

Thenewtoolprovidesthefunctionofcodecoverageanalysisforthekernel.Coverageanalysisshowswhichlinesofcodeinthekernelareexecutedwhenagiventestisrun.Moreimportantly,thecoverageanalysisindicateswhichpartsofthekernelhavenotbeentestedatall.Thisdataisimportantbecauseitpointsoutwhichnewtestsneedtobewrittentotestthosepartsofthekernelsothatthekernelcangetamorecompletetest.

Alotofinformation

Intheprocessofdevelopingthefuture2.6Linuxkernel,inadditiontotheseautomatedinformationmanagementmethods,differentmembersoftheopensourcecommunityalsocollectandtrackAstaggeringamountofinformation.

Forexample,astatuslistwascreatedontheKernelNewbiessitetokeeptrackofthenewcomponentsofthekernelthathavebeenproposed.Thislistcontainsentriessortedbystatus.Iftheyhavebeencompleted,theyareincludedinwhichkernel,andiftheyhavenotbeencompleted,itwilltakehowlong.LinkstomanyitemsonthelistpointtotheWebsiteofalargeproject,orwhentheitemsaresmall,thelinkpointstoacopyofthee-mailmessageexplainingthecorrespondingpart.

Testing

Applicationtesting

Testbackground

Inthepast,LinuxkerneltestingmethodsrevolvedaroundopensourcecodeThedevelopmentmodeliscarriedout.Sincethecodeisreleasedtootherdevelopersforreviewonceitisreleased,therehasneverbeenaformalverificationcyclesimilartootherformsofsoftwaredevelopment.Thetheoreticalbasisbehindthismethodistheso-called"Linuslaw"in"TheCathedralandtheBazaar"(pleaserefertoreferencesforrelevantreferences).Thecontentofthislawis"people'seyesarediscerning."Inotherwords,aintensivereviewcanidentifymostoftherealbigproblems.

However,inreality,thekernelhasmanycomplexinterconnections.Evenwithsufficientreview,manyseriousflawswillstillbemissed.Inaddition,oncethelatestkernelisreleased,enduserscan(andoftendo)downloadanduseit.When2.4.0wasreleased,manypeopleinthecommunityproposedmoreorganizedteststoensuretheintensityofspecifictestsandcodereviews.Organizedtestingincludestheuseoftestplans,repeatabilityinthetestingprocess,andsoon.Usingallthreemethodswillresultinhighercodequalitythanusingonlytwomethodsinitially.

TestProject

TheearliestcontributortotheorganizedtestingofLinuxwastheLinuxTestProject(LinuxTestProject,LTP).ThepurposeofthisprojectistoimprovethequalityofLinuxthroughmoreorganizedtestingmethods.Partofthistestprojectisthedevelopmentofautomatedtestsuites.ThemaintestsuitedevelopedbyLTPisalsocalledLinuxtestproject.Whenthe2.4.0kernelwasreleased,theLTPtestsuitehadonlyabout100tests.Withthedevelopmentandmaturityofthe2.4and2.5versionsofLinux,theLTPtestsuiteisalsodevelopingandmaturing.Currently,theLinuxtestprojectincludesmorethan2,000tests,andthisnumberisstillgrowing.

Regressiontesting

Inthe2.5developmentcycle,anotheritemusedintheLinuxtestprojectistousetheLTPtestsuitetoperformregressiontestingontheLinuxkernelformultipledays.PeopleuseBitKeepertocreateareal-time,centralizedarchivetogetasnapshotoftheLinuxkernelatanytime.WhenBitKeeperandsnapshotswerenotused,testershadtowaituntilthekernelwasreleasedbeforetheycouldstarttesting.Aslongasthekernelischanged,thetestercanperformthetest.

Anotheradvantageofusingautomatedtoolstoperformregressiontestingthatlastsformanydaysisthatthechangesaresmallcomparedtotheprevioustest.Ifanewregressiondefectisfound,itisusuallyeasytodetectwhichchangethedefectmayhavecaused.

Similarly,becauseitisthelatestchange,itstillhasadeepimpressioninthedeveloper'smind-hopethiswillmakeiteasierforthemtorememberandrevisethecorrespondingcode.PerhapstheLinusLawshouldhavesuchaconclusionthatsomedefectsareeasiertofindthanothers,becausethosearetheonesdiscoveredanddealtwithbythekernelregressionteststhatlastedformanydays.Thesetestscanbeperformeddailyduringthedevelopmentcycleandbeforetheactualrelease,whichallowstesterswhoonlyfocusonthefullreleaseversiontofocusonmoreseriousandtime-consumingdefects.

Extensibletesting

AnotherteamcalledOpenSourceDevelopmentLabs(OSDL)hasalsomadeimportantcontributionstoLinuxtesting.Soonafterthe2.4kernelwasreleased,OSDLcreatedasystemcalledScalableTestPlatform(STP).STPisanautomatedtestplatformthatallowsdevelopersandtesterstorunthetestsprovidedbythesystemonOSDLhardware.Developerscanevenusethissystemtotesttheirownkernel-specificpatches.Theextensibletestplatformsimplifiestheteststeps,becauseSTPcanbuildthekernel,setupthetest,runthetest,andcollecttheresults.Thengettheresultsforin-depthcomparison.Manypeoplecannotaccesslargesystems,suchasSMPmachineswith8processors.ThroughSTP,anyonecanruntestsonlargesystemslikethis.Anotheradvantageofthissystem(STP)liesinthis.

Trackingdefects

Sincethereleaseof2.4,oneofthebiggestimprovementstotheorganizedtestingoftheLinuxkernelisdefecttracking.Inthepast,defectsfoundintheLinuxkernelwerereportedtotheLinuxkernelmailinglist,tothemailinglistofspecificcomponentsorspecificsystems,ordirectlytotheindividualwhomaintainedthepartofthecodewherethedefectwasfound.WiththeincreaseinthenumberofpeopledevelopingandtestingLinux,theshortcomingsofthissystemwerequicklyexposed.Inthepast,unlesspeople'sreportsondefectscouldbemaintainedsurprisingly,defectswereoftenmissed,forgottenorignored.

OSDLhasinstalledadefecttrackingsystemtoreportandtrackdefectsintheLinuxkernel.Thesystemisconfiguredsothatwhenacomponent’sdefectisreported,themaintainerofthatcomponentwillbenotified.Themaintainercaneitheracceptandfixthedefect,orreassignthedefect(ifitisfinallydeterminedthatitisactuallyadefectinanotherpartofthekernel),orexcludeit(ifitisfinallydeterminedthatitisnotarealdefect,suchasamisconfiguredsystem).Thedefectsreportedtothemailinglistarealsoindangerofbeinglost,asmoreandmoree-mailsarefloodingtothatlist.However,inthedefecttrackingsystem,thereisalwaysarecordofeachdefectanditscurrentstatus.

Related Articles
TOP