Reasons
ThelinklayerhasthefeatureofmaximumtransmissionunitMTU,whichlimitsthemaximumlengthofdataframes,anddifferentnetworktypeshaveanupperlimit.TheMTUofEthernetis1500,youcanusethenetstat-icommandtocheckthisvalue.IfthereisadatapackettobetransmittedattheIPlayerandthelengthofthedatapacketexceedstheMTU,theIPlayerwillfragmentthedatapacketsothatthelengthofeachfragmentislessthanorequaltotheMTU.WeassumethataUDPdatapacketistobetransmitted,theMTUofEthernetis1500bytes,thegeneralIPheaderis20bytes,theUDPheaderis8bytes,andthepayloadpartofthedataisreservedfor1500-20-8=1472bytes.Ifthedatapartislargerthan1472bytes,fragmentationwilloccur.
Detailedintroduction
Shardingisatypeofdatabasepartitioning,whichdivideslargedatabasesintosmaller,fasterandeasiertomanageparts.Thesepartsarecalleddatafragments..Thewordfragmentmeansasmallpartofthewhole.
JasonTeesaid:"Inshort,ashardingdatabaseneedstodividethedatabaseintomultiplesmalldatabasesthathavenocommonground,andtheycanbespreadacrossmultipleservers."
p>Technicallyspeaking,shardingissynonymouswithhorizontalpartitioning.Inpractice,thistermisoftenusedtorefertoalldatabasepartitionsthatmakealargedatabaseeasiertomanage.
Thecoreconceptofshardingisbasedonanidea:thesizeofthedatabaseandthenumberoftransactionsperunittimeonthedatabaseincreaselinearly,andtheresponsetimeofqueryingthedatabaseincreasesexponentially.
Inaddition,thecostofcreatingandmaintainingalargedatabaseinoneplacewillincreaseexponentially,becausethedatabasewillrequirehigh-endcomputers.Conversely,datafragmentscanbedistributedtoalargenumberofmuchcheapercommercialservers.Intermsofhardwareandsoftwarerequirements,datafragmentationisrelativelyunrestricted.
Insomecases,databaseshardingcanbedonesimply.Splittingtheuserdatabasebygeographiclocationisacommonexample.Usersontheeastcoastareassignedtooneserver,andusersonthewestcoastareassignedtoanotherserver.Assumingthatnouserhasmultiplegeographiclocations,thistypeofpartitioniseasytomaintainandcreaterules.
Butdatashardingcanbeamorecomplicatedprocessinsomecases.Forexample,ifadatabaseholdsverylittlestructureddata,shardingmaybeverycomplicated,andtheresultingfragmentationmaybedifficulttomaintain.
Fragmentationprocess
ForeachIPdatagramsentbythesender,itsidentificationfieldcontainsavalue.Thisvalueiscopiedtoeachfragmentwhenthedatagramisfragmented.Theflagfieldusesonebittoindicate"moreslices".Exceptforthelastslice,everyotherslicethatcomposesthedatagrammustsetthisbitto1.Thefragmentoffsetfieldreferstothepositionofthefragmentoffsetfromthebeginningoftheoriginaldatagram.Inaddition,whenthedatagramisfragmented,thetotallengthofeachfragmentshouldbechangedtothelengthofthefragment.
Finally,thereisabitintheflagfieldcalledthe"non-fragmented"bit.Ifthisbitissetto1,IPwillnotfragmentthedatagram.Instead,thedatagramisdiscardedandanICMPerrormessageissenttotheinitiator.
AftertheIPdatagramisfragmented,eachfragmentbecomesapacketwithitsownIPheader,andisindependentofotherpacketswhenselectingaroute.Inthisway,thesepiecesofdatagrammaybeoutofsequencewhentheyarriveatthedestination,butthereisenoughinformationintheIPheaderforthereceivingendtoassemblethesepiecesofdatagramcorrectly.
Technicalinformation
TheIPheadercontainstheinformationneededforfragmentationandreorganization:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Identification|R|DF|MF|FragmentOffset|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
||||
Identification:TheidentificationfieldoftheIPdatapacketsentbythesenderisasinglevalue,whichiscopiedtoeachsliceduringfragmentation.
R:Keepunused.
DF:Don'tFragment,"Donotfragment"bit.Ifthisbitissetto1,theIPlayerwillnotfragmentthedatagram.
MF:MoreFragment,"morefragments",exceptforthelastfragment,thisbitmustbesetto1foreveryotherfragmentthatcomposesthedatagram.
FragmentOffset:Thepositionofthefragmentoffsetfromthebeginningoftheoriginaldatapacket.Thenumberofbytesoftheoffsetisthisvaluemultipliedby8.
Inaddition,whenthedatagramisfragmented,thetotallengthofeachfragmentshouldbechangedtothelengthofthefragment.