- 浏览: 208620 次
- 性别:
- 来自: 杭州
最新评论
-
李嘉图0624:
第一段代码就出错了,少了g.close();不关闭的话,也没有 ...
Jackson 入门 -
daichangfu:
不光是生僻字需要,正常的也需要啊。
解决Java应用在Linux下无法正常水印生僻字 -
yaoweinan:
不错
JGroup配置笔记 -
learnworld:
grandboy 写道我想json的数据多一些对于数据绑定回j ...
Jackson 入门 -
grandboy:
我想json的数据多一些对于数据绑定回javabean时不会出 ...
Jackson 入门
Our application need to meet the reliable communication and the least data loss requirements for the JGroup cluster. The JGroup-2.6.1 release provides some out-of-box configurations in xml format. I choose the tcp-nio.xml configuration, because the inefficiency of the traditional IO model and unreliable data transmission of UDP. In this article I will try to explain why we prefer TCP-NIO method and then the configuration details from tcp-nio.xml.
1
1.1
Why TCP-NIO?
Traditional I/O classes have primarily been stream-oriented, often invoking methods on several layers of objects to handle individual bytes or characters.
This object-oriented approach, composing behaviors by plugging I/O objects together, offering tremendous flexibility but can be a performance killer when large amounts of data must be handled. Efficiency is the goal of I/O, and efficient I/O often doesn't map well to objects. Efficient I/O usually means that you must take the shortest path from Point A to Point B. Complexity destroys performance when doing high-volume I/O.
The traditional I/O abstractions of the Java platform have served well and are appropriate for a wide range of uses. But these classes do not scale well when moving large amounts of data, nor do they provide some common I/O functionality widely available on most operating systems today. These features — such as file locking, non-blocking I/O, readiness selection, and memory mapping — are essential for scalability and may be required to interact properly with non-Java applications, especially at the enterprise level. The classic Java I/O mechanism doesn't model these common I/O services.
Java Specification Request#51 (http://jcp.org/jsr/detail/51.jsp) details the need for high-speed, scalable I/O, which better leverages the I/O capabilities of the underlying operating system.
—from O’Reilly Java NIO
1.2
Why not UDP?
Our application might need to go over WAN, and there is chance that some routers over become overloaded which result in discarding packets. The Order the router drops a packet is Broadcast, Multicast, and finally Unicast.
2
2.1 Overview
JGroups uses a JChannel as the main API to connect to a cluster, send and receive messages, and to register listeners that are called when things (such as member joins) happen.
What is sent around are Messages, which contain a byte buffer (the payload), plus the sender's and receiver's address. Addresses are subclasses of org.jgroups.Address, and usually contain an IP address plus a port.
To join a cluster, we'll use a JChannel. An instance of JChannel is created with a configuration (e.g. an XML file) which defines the properties of the channel.
The protocol stack contains a number of protocol layers in a bidirectional list. All messages sent and received over the channel have to pass through the protocol stack. Every layer may modify, reorder, pass or drop a message, or add a header to a message.
The protocol stack in the xml configuration file outlines the protocols bottom-up, take tcp-nio,xml for example, bottom protocol is TCP_NIO and the upmost protocol is pbcast.STREAMING_STATE_TRANSFER. The order shouldn’t be changed.
2.2
Let’s take a look at the configuration:
1) TCP is a replacement of UDP as bottom layer in cases where IP Multicast based on UDP is not desired. This may be the case when operating over a WAN, where routers will discard IP MCAST. As a rule of thumb UDP is used as transport for LANs, whereas TCP is used for WANs. Whereas TCPNIO is a replacement of TCP, TCP_NIO performs much better because it fully take advantage of the NIO services.
2) TCPPING The TCPPING protocol layer retrieves the initial membership in answer to the GMS's FIND_INITIAL_MBRS event.
3) MERGE2 If a group gets split for some reasons (e.g. network partition), this protocol merges the subgroups back into one group.
4) FD_SOCK Failure detection protocol based on a ring of TCP sockets created between group members.
5) FD Failure detection based on heartbeat messages. If reply is not received without timeout ms, max_tries times, a member is declared suspected, and will be excluded by GMS. If we use FD_SOCK instead, then we don't send heartbeats, but establish TCP sockets and declare a member dead only when a socket is closed.
6) VERIFY_SUBSPECT Verifies that a suspected member is really dead by pinging that member once again. Drops suspect message if member does respond. Tries to minimize false suspicions.
7) pbcast.NAKACK Lossless and FIFO delivery of multicast messages, using negative acks. E.g. when receiving P:1, P:3, P:4, a receiver asks P for retransmission of message 2.
8) VIEW_SYNC Periodically sends the view to the group. When a view is received which is greater than the current view, we install it. Otherwise we simply discard it. This is used to solve the problem for unreliable view dissemination outlined in JGroups/doc/ReliableViewInstallation.txt. This protocol is supposed to be just below GMS.
9) pbcast.GMS Group Membership Service. Responsible for joining/leaving members. Also handles suspected members, and excludes them from the membership. Sends Views (topology configuration) to all members when a membership change has occurred.
10) FC Credit based flow control protocol. When senders send faster than receivers can process the messages, over time the receivers will run out of memory. Therefore the senders have to be throttled to the fastest rate at which all receivers can process messages.
11) FRAG2 Fragments messages larger than frag_size bytes
12) STREAMING_STATE_TRANSFER In order to transfer application state to a joining member of a group pbcast.STATE_TRANSFER has to load entire
state into memory and send it to a joining member. Major limitation of this approach is that the state transfer that isvery large (>1Gb) would likely result in OutOfMemoryException. In order to alleviate this problem a new state transfer methodology, based on a streaming state transfer
Details descriptions of each protocols and its configuration parameter can be found on JGroup WIKI
2.2 Notes on some important parameters:
2.2.1 bind_addr on TCP-NIO
On which address JGroup to create the server socket. Though optional, it’s Useful in case of a multi-homed machine and you just want it bind on the specific address. If not specified, the server socket will be created on all available interfaces
2.2.2 initial_hosts on TCPPING
The comma-separated lists of hosts names and ports used to connect to get the initial memberships. Note that there should not be spaces between the entries in the list. TCPPING.initial_hosts should contain the universe of all possible members. If you decide to add a new member who isn't listed in TCP.initial_hosts, that node will still be able to join, but you may have the problem
For example, there are 192.168.1.1, 192.168.1.2, 192.168.1.3 and 192.168.1.4 starting at port 7800 in the cluster:
Your TCPPING’s configuration should be :
All the other protocols and their respective parameters can remain untouched.
1
1.1
Why TCP-NIO?
Traditional I/O classes have primarily been stream-oriented, often invoking methods on several layers of objects to handle individual bytes or characters.
This object-oriented approach, composing behaviors by plugging I/O objects together, offering tremendous flexibility but can be a performance killer when large amounts of data must be handled. Efficiency is the goal of I/O, and efficient I/O often doesn't map well to objects. Efficient I/O usually means that you must take the shortest path from Point A to Point B. Complexity destroys performance when doing high-volume I/O.
The traditional I/O abstractions of the Java platform have served well and are appropriate for a wide range of uses. But these classes do not scale well when moving large amounts of data, nor do they provide some common I/O functionality widely available on most operating systems today. These features — such as file locking, non-blocking I/O, readiness selection, and memory mapping — are essential for scalability and may be required to interact properly with non-Java applications, especially at the enterprise level. The classic Java I/O mechanism doesn't model these common I/O services.
Java Specification Request#51 (http://jcp.org/jsr/detail/51.jsp) details the need for high-speed, scalable I/O, which better leverages the I/O capabilities of the underlying operating system.
—from O’Reilly Java NIO
1.2
Why not UDP?
Our application might need to go over WAN, and there is chance that some routers over become overloaded which result in discarding packets. The Order the router drops a packet is Broadcast, Multicast, and finally Unicast.
2
2.1 Overview
JGroups uses a JChannel as the main API to connect to a cluster, send and receive messages, and to register listeners that are called when things (such as member joins) happen.
What is sent around are Messages, which contain a byte buffer (the payload), plus the sender's and receiver's address. Addresses are subclasses of org.jgroups.Address, and usually contain an IP address plus a port.
To join a cluster, we'll use a JChannel. An instance of JChannel is created with a configuration (e.g. an XML file) which defines the properties of the channel.
The protocol stack contains a number of protocol layers in a bidirectional list. All messages sent and received over the channel have to pass through the protocol stack. Every layer may modify, reorder, pass or drop a message, or add a header to a message.
The protocol stack in the xml configuration file outlines the protocols bottom-up, take tcp-nio,xml for example, bottom protocol is TCP_NIO and the upmost protocol is pbcast.STREAMING_STATE_TRANSFER. The order shouldn’t be changed.
2.2
Let’s take a look at the configuration:
<config> <TCP_NIO bind_addr="localhost" start_port="7800" loopback="true" recv_buf_size="20000000" send_buf_size="640000" discard_incompatible_packets="true" max_bundle_size="64000" max_bundle_timeout="30" use_incoming_packet_handler="true" enable_bundling="true" use_send_queues="false" sock_conn_timeout="300" skip_suspected_members="true" use_concurrent_stack="true" thread_pool.enabled="true" thread_pool.min_threads="1" thread_pool.max_threads="25" thread_pool.keep_alive_time="5000" thread_pool.queue_enabled="false" thread_pool.queue_max_size="100" thread_pool.rejection_policy="Run" oob_thread_pool.enabled="true" oob_thread_pool.min_threads="1" oob_thread_pool.max_threads="8" oob_thread_pool.keep_alive_time="5000" oob_thread_pool.queue_enabled="false" oob_thread_pool.queue_max_size="100" oob_thread_pool.rejection_policy="Run" reader_threads="3" writer_threads="3" processor_threads="0" processor_minThreads="0" processor_maxThreads="0" processor_queueSize="100" processor_keepAliveTime="9223372036854775807"/> <TCPPING timeout="3000" initial_hosts="${jgroups.tcpping.initial_hosts:localhost[7800],localhost[7801]}" port_range="1" num_initial_members="3"/> <MERGE2 max_interval="100000" min_interval="20000"/> <FD_SOCK/> <FD timeout="10000" max_tries="5" shun="true"/> <VERIFY_SUSPECT timeout="1500" /> <pbcast.NAKACK use_mcast_xmit="false" gc_lag="0" retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true"/> <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000"/> <VIEW_SYNC avg_send_interval="60000"/> <pbcast.GMS print_local_addr="true" join_timeout="3000" shun="true" view_bundling="true"/> <FC max_credits="2000000" min_threshold="0.10"/> <FRAG2 frag_size="60000" /> <pbcast.STREAMING_STATE_TRANSFER/> </config>
1) TCP is a replacement of UDP as bottom layer in cases where IP Multicast based on UDP is not desired. This may be the case when operating over a WAN, where routers will discard IP MCAST. As a rule of thumb UDP is used as transport for LANs, whereas TCP is used for WANs. Whereas TCPNIO is a replacement of TCP, TCP_NIO performs much better because it fully take advantage of the NIO services.
2) TCPPING The TCPPING protocol layer retrieves the initial membership in answer to the GMS's FIND_INITIAL_MBRS event.
3) MERGE2 If a group gets split for some reasons (e.g. network partition), this protocol merges the subgroups back into one group.
4) FD_SOCK Failure detection protocol based on a ring of TCP sockets created between group members.
5) FD Failure detection based on heartbeat messages. If reply is not received without timeout ms, max_tries times, a member is declared suspected, and will be excluded by GMS. If we use FD_SOCK instead, then we don't send heartbeats, but establish TCP sockets and declare a member dead only when a socket is closed.
6) VERIFY_SUBSPECT Verifies that a suspected member is really dead by pinging that member once again. Drops suspect message if member does respond. Tries to minimize false suspicions.
7) pbcast.NAKACK Lossless and FIFO delivery of multicast messages, using negative acks. E.g. when receiving P:1, P:3, P:4, a receiver asks P for retransmission of message 2.
8) VIEW_SYNC Periodically sends the view to the group. When a view is received which is greater than the current view, we install it. Otherwise we simply discard it. This is used to solve the problem for unreliable view dissemination outlined in JGroups/doc/ReliableViewInstallation.txt. This protocol is supposed to be just below GMS.
9) pbcast.GMS Group Membership Service. Responsible for joining/leaving members. Also handles suspected members, and excludes them from the membership. Sends Views (topology configuration) to all members when a membership change has occurred.
10) FC Credit based flow control protocol. When senders send faster than receivers can process the messages, over time the receivers will run out of memory. Therefore the senders have to be throttled to the fastest rate at which all receivers can process messages.
11) FRAG2 Fragments messages larger than frag_size bytes
12) STREAMING_STATE_TRANSFER In order to transfer application state to a joining member of a group pbcast.STATE_TRANSFER has to load entire
state into memory and send it to a joining member. Major limitation of this approach is that the state transfer that isvery large (>1Gb) would likely result in OutOfMemoryException. In order to alleviate this problem a new state transfer methodology, based on a streaming state transfer
Details descriptions of each protocols and its configuration parameter can be found on JGroup WIKI
2.2 Notes on some important parameters:
2.2.1 bind_addr on TCP-NIO
On which address JGroup to create the server socket. Though optional, it’s Useful in case of a multi-homed machine and you just want it bind on the specific address. If not specified, the server socket will be created on all available interfaces
2.2.2 initial_hosts on TCPPING
The comma-separated lists of hosts names and ports used to connect to get the initial memberships. Note that there should not be spaces between the entries in the list. TCPPING.initial_hosts should contain the universe of all possible members. If you decide to add a new member who isn't listed in TCP.initial_hosts, that node will still be able to join, but you may have the problem
For example, there are 192.168.1.1, 192.168.1.2, 192.168.1.3 and 192.168.1.4 starting at port 7800 in the cluster:
Your TCPPING’s configuration should be :
<TCPPING timeout="3000" initial_hosts="${jgroups.tcpping.initial_hosts: 192.168.1.1 [7800], 192.168.1.2 [7800], 192.168.1.3 [7800], 192.168.1.4 [7800]}" port_range="1" num_initial_members="4"/>
All the other protocols and their respective parameters can remain untouched.
发表评论
-
开发新手通用代码问题
2011-06-04 06:32 13168最近辅导黄埔的同学进行开发,发现了一些问题,隐约看到当年自己写 ... -
jmockit 的使用系列
2010-12-13 21:30 94171. 入门 2. 完整的Mock步骤 ... -
jmockit 的使用示例-02完整的Mock步骤
2010-12-13 21:11 4875package jmockit.sample; impo ... -
jmockit 的使用示例-03全部mock的?
2010-12-13 21:11 3644package jmockit.sample; impo ... -
jmockit 的使用示例-04静态部分mock示例
2010-12-13 21:10 3862package jmockit.sample; impo ... -
jmockit 的使用示例-06私有成员的Mock
2010-12-13 21:10 7652目标测试代码 package jm ... -
jmockit 的使用示例-07私有静态成员Mock
2010-12-13 21:09 4690目标测试代码 package jmockit.target ... -
jmockit 的使用示例-09构造方法也可以Mock?基于状态的Mock?
2010-12-13 21:07 10376目标测试代码 package jmockit.target ... -
jmockit 的使用示例-10另一种基于状态的Mock,随穿随脱?
2010-12-13 21:07 2705目标测试代码 package jm ... -
jmockit 的使用示例-11不是吧,还能访问实际被Mock的对象?
2010-12-13 21:07 3231目标测试代码 package jmockit.target ... -
jmockit 的使用示例-07私有静态成员Mock
2010-12-13 20:40 5889目标测试代码 package jmockit.target ... -
jmockit 的使用示例-02完整的Mock步骤
2010-12-13 20:31 1462package jmockit.sample; impo ... -
开放的心态
2010-10-02 13:56 981今天看李笑来的《把时间当作朋友》,看到一句话,“我们的大脑 ... -
Jackson 入门
2010-09-15 07:17 6492同事的一些测试结果看来,Jackson在处理Json方面性能相 ... -
解决Java应用在Linux下无法正常水印生僻字
2010-04-10 13:46 5616昨天接到一个投诉,说是他的水印是一个方框。刚开始还以为是程序出 ... -
同学们,秒投简历啦!
2010-03-12 23:51 221公司业务急速扩张,急需有才能的同学加入到我们的行列中, ... -
反编译工具
2009-12-31 20:54 1132追查JVM崩溃过程中使用到的反编译工具,其中beanutils ... -
追查JVM崩溃
2009-12-31 20:25 2240JDK升级到1.6后,服务器集群经常崩溃,后来把VM的参 ... -
普通用户下实现Apache 2.2.14与jboss-5.1.0.GA集成
2009-11-29 23:05 2766Apache与jboss集成,实际是apache与tomcat ... -
swt 笔记
2009-11-27 00:24 891http://www.ibm.com/developerwor ...
相关推荐
jgroup详细介绍
Reliable group communication with JGroups 3.x Preface This is the JGroups manual. It provides information about: ...The focus is on how to use JGroups, not on how JGroups is implemented.
jgroup代码
EHCAHCE基于JGROUP的集群配置方案,内含相关配置文件,及配置说明
NULL 博文链接:https://8366.iteye.com/blog/921760
jgroup笔记.
jroup是一个比较优秀的集群通讯开源软件,本实例展示如何用jgroup进行不同机器之间的通讯
找不到对方在调试的时候发现只要是使用了SWT的类的地方会出现线程错误,于是我想是不是出现了线程同步的问题经询问别人后得知在SWT中使用JGroup应该要使线程同步,应该使用Display类的syncExec(Runnable r)方法于是...
JGroup可以基于TCP协议来实现消息广播,也可以通过UDP方式来广播消息,利弊不言而喻,TCP可靠,但是代价大,性能没有UDP来的好,UDP速度快,代价小,但是消息的丢失率以及无序性有着很大的限制。但是JGroup在UDP方式...
最强大得UUP开源组件,用于底层通讯,以被JBOSS采用
jboss jdbc json jgroup.jar
jgroup 源码 HIBERNATE 二级缓存 集群
jgroup配置[收集].pdf
jgroup配置[归类].pdf
它基于IP多播(也支持TCP),但是有一些特殊功能,例如可靠性和组成员身份。 在这个项目中,我通过使用JGroups作为消息传递工具实现了分布式(复制)堆栈和集合。 这些复制的数据结构将使用状态转移来处理新成员...
jgroups官方帮助文档html格式2.X版本
NULL 博文链接:https://hxy-go.iteye.com/blog/518795
NULL 博文链接:https://wiselyman.iteye.com/blog/2114715
jGroup-OOPProjekt rocc 包含整个项目。 在 rocc 中还包含两个 bat 文件。 “reBuild”文件重建项目,另一个“run”运行项目。 RuinsOfCorrosaCity.zip 是可玩游戏及其脚本的包。 文档包含 RAD 和 SDD 会议包含...